2024-04-24T22:57:22.540 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T22:57:22.541 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_teuthology@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/fs/norstats}} --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181 -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/orig.config.yaml 2024-04-24T22:57:22.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:57:22.700 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/fs/norstats}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672181' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client dirsize rbytes: false client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync rbytes: false syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgketQo4ZgEuVU0R4iPgbWtoeblupcpIM3M0pXwHpNfqDoLrOMlID0zrd0A+s9jSGDKZr4D2yqIY5pb3fgRnfU= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJ2sKRG7vMbYPLRRqqaEG60XHWHcSqpSdQ1sCn6MLcTKOnMwmfaWhi8QbeT750VXf3eBVizHc+QznLZioELWqQ= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDxjzDT71ET55Iay1bgozTLkPDRSXFU/7f/vYQJ/bes0PG//QiL/rVIXNkMJWMM2jSZuzsEjDkSFYFIqaTezCfI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - fs/norstats teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T22:57:22.700 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T22:57:22.701 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T22:57:22.701 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T22:57:22.702 INFO:teuthology.task.internal:Checking packages... 2024-04-24T22:57:22.728 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T22:57:22.728 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:57:22.728 INFO:teuthology.packaging:ref: None 2024-04-24T22:57:22.728 INFO:teuthology.packaging:tag: None 2024-04-24T22:57:22.728 INFO:teuthology.packaging:branch: squid 2024-04-24T22:57:22.728 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:57:22.728 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T22:57:23.044 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T22:57:23.046 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T22:57:23.077 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T22:57:23.077 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T22:57:23.095 INFO:teuthology.task.internal:Saving configuration 2024-04-24T22:57:23.122 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T22:57:23.141 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T22:57:23.173 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:52:04.790169', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDgketQo4ZgEuVU0R4iPgbWtoeblupcpIM3M0pXwHpNfqDoLrOMlID0zrd0A+s9jSGDKZr4D2yqIY5pb3fgRnfU='} 2024-04-24T22:57:23.200 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:52:04.792972', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJ2sKRG7vMbYPLRRqqaEG60XHWHcSqpSdQ1sCn6MLcTKOnMwmfaWhi8QbeT750VXf3eBVizHc+QznLZioELWqQ='} 2024-04-24T22:57:23.227 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:52:04.791716', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDxjzDT71ET55Iay1bgozTLkPDRSXFU/7f/vYQJ/bes0PG//QiL/rVIXNkMJWMM2jSZuzsEjDkSFYFIqaTezCfI='} 2024-04-24T22:57:23.227 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T22:57:23.252 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T22:57:23.252 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T22:57:23.252 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T22:57:23.252 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T22:57:23.392 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f456dcfe160>, signals=[15]) 2024-04-24T22:57:23.392 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T22:57:23.402 INFO:teuthology.task.internal:Opening connections... 2024-04-24T22:57:23.404 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-24T22:57:23.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:57:23.533 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-04-24T22:57:23.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:57:23.617 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-04-24T22:57:23.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:57:23.691 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T22:57:23.700 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-24T22:57:23.719 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-24T22:57:23.720 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:57:23.777 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:57:23.778 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:57:23.778 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:57:23.778 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:57:23.778 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-24T22:57:23.801 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-04-24T22:57:23.819 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-04-24T22:57:23.820 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-04-24T22:57:23.878 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-04-24T22:57:23.878 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-04-24T22:57:23.878 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-04-24T22:57:23.878 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:57:23.878 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:57:23.879 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:57:23.879 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-04-24T22:57:23.900 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-04-24T22:57:23.918 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-04-24T22:57:23.919 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:57:23.976 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:57:23.977 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:57:23.977 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:57:23.977 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:57:23.977 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-04-24T22:57:24.000 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T22:57:24.011 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T22:57:24.062 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T22:57:24.063 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:57:24.066 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:57:24.068 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:57:24.085 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T22:57:24.090 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T22:57:24.091 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:57:24.122 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:57:24.124 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:57:24.445 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T22:57:24.514 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T22:57:24.531 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T22:57:24.531 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-24T22:57:24.532 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-24T22:57:24.532 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:57:24.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:24.549 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-04-24T22:57:24.607 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:57:24.607 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-437.el9.x86_64 2024-04-24T22:57:24.608 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-04-24T22:57:28.107 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.0 MB 00:01 2024-04-24T22:57:29.816 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-04-24T22:57:35.675 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.2 MB 00:02 2024-04-24T22:57:37.575 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-24T22:57:39.518 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-24T22:57:44.651 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-24T22:57:46.482 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:57:46.483 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:57:46.568 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-24T22:57:46.569 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-04-24T22:57:46.569 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-24T22:57:46.671 DEBUG:teuthology.orchestra.run.smithi112:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:57:47.169 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:57:44 PM UTC. 2024-04-24T22:57:47.287 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-24T22:57:47.295 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-24T22:57:47.295 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-04-24T22:57:47.295 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-24T22:57:47.295 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalling: 2024-04-24T22:57:47.295 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:57:47.295 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T22:57:47.295 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-24T22:57:47.295 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-24T22:57:47.296 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T22:57:47.296 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 255 k 2024-04-24T22:57:47.296 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 0 2024-04-24T22:57:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:Operation aborted. 2024-04-24T22:57:47.347 INFO:teuthology.orchestra.run.smithi112.stdout:Is this ok [y/N]: 2024-04-24T22:57:47.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum reinstall -y kernel || true 2024-04-24T22:57:47.840 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:57:44 PM UTC. 2024-04-24T22:57:47.955 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-24T22:57:47.956 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-24T22:57:47.956 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-04-24T22:57:47.956 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-24T22:57:47.956 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalling: 2024-04-24T22:57:47.956 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:57:47.956 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T22:57:47.956 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-24T22:57:47.956 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-24T22:57:47.957 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T22:57:47.957 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 255 k 2024-04-24T22:57:47.957 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 0 2024-04-24T22:57:47.957 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-24T22:57:48.895 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-437.el9.x86_64.rpm 382 kB/s | 255 kB 00:00 2024-04-24T22:57:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:57:48.896 INFO:teuthology.orchestra.run.smithi112.stdout:Total 271 kB/s | 255 kB 00:00 2024-04-24T22:57:48.920 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-24T22:57:49.109 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-24T22:57:49.119 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-24T22:57:49.119 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-24T22:57:49.119 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-24T22:57:49.244 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-24T22:57:49.313 INFO:teuthology.orchestra.run.smithi112.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:57:49.313 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:49.975 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:49.975 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:57:50.705 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:50.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T22:57:50.705 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalled: 2024-04-24T22:57:50.705 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:57:50.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T22:57:50.705 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-24T22:57:50.921 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:57:50.967 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:57:50.968 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:57:50.968 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:57:50.968 DEBUG:teuthology.parallel:result is None 2024-04-24T22:57:50.968 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T22:57:50.979 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T22:57:50.979 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:57:50.983 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:57:50.985 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:57:51.003 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T22:57:51.010 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T22:57:51.016 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T22:57:51.017 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:57:51.038 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:57:51.043 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:57:51.142 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T22:57:51.149 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T22:57:51.149 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:57:51.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:51.166 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:57:51.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:51.183 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:57:51.186 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:57:51.210 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:57:51.214 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:51.241 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:51.246 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:51.267 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:51.279 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:51.284 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:51.285 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T22:57:51.293 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T22:57:51.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:57:51.296 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:57:51.324 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:57:51.354 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T22:57:51.363 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T22:57:51.363 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:57:51.366 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:57:51.392 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:57:51.412 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:51.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:51.513 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:51.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:51.635 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T22:57:51.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:57:51.699 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:51.741 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:51.794 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:51.874 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:51.920 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T22:57:51.920 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:57:51.986 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:52.027 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:52.084 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:52.165 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:52.209 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T22:57:52.210 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:57:52.274 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-24T22:57:52.277 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-04-24T22:57:52.279 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-04-24T22:57:52.310 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:57:52.314 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:57:52.353 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:57:52.816 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T22:57:52.825 INFO:teuthology.task.internal:Starting timer... 2024-04-24T22:57:52.825 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T22:57:52.834 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T22:57:52.842 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T22:57:52.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-04-24T22:57:52.927 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-04-24T22:57:52.929 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-04-24T22:57:52.989 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-04-24T22:57:52.991 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-04-24T22:57:53.056 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-04-24T22:57:53.058 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T22:57:53.059 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-04-24T22:57:53.078 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-04-24T22:57:53.079 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-04-24T22:57:53.103 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-04-24T22:57:53.105 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-04-24T22:57:53.129 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-04-24T22:57:53.129 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2024-04-24T22:57:53.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:57:53.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:53.156 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:57:53.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:53.183 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:57:53.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:53.210 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T22:57:53.210 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:57:53.234 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:57:53.260 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:57:53.290 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T22:57:53.299 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T22:57:53.304 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T22:57:53.434 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T22:57:53.451 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T22:57:53.452 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi150.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T23:01:53.969 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-04-24T23:01:53.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-24T23:01:53.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:01:54.048 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-24T23:01:54.125 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-24T23:01:54.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-24T23:01:54.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:01:54.200 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-04-24T23:01:54.277 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-04-24T23:01:54.277 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-24T23:01:54.278 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:01:54.352 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-24T23:01:54.435 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-24T23:01:54.435 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T23:01:54.445 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T23:01:54.446 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:01:54.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:01:54.448 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:01:54.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:01:54.450 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:01:54.451 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:01:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:01:54.476 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:01:54.488 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:01:54.489 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:01:54.507 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:01:54.511 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-04-24T23:01:54.512 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-04-24T23:01:54.521 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-04-24T23:01:54.522 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:01:54.522 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-04-24T23:01:54.534 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:01:54.535 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:01:54.546 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-04-24T23:01:54.547 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:01:54.548 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:01:54.557 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-04-24T23:01:54.571 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:01:54.585 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:01:54.600 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:01:54.603 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:01:54.604 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-24T23:01:54.604 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.604 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.604 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.604 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.606 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:01:54.609 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:01:54.609 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-24T23:01:54.610 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.610 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.610 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.610 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.640 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:01:54.644 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:01:54.644 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-04-24T23:01:54.644 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.644 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.644 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.644 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:01:54.645 INFO:teuthology.run_tasks:Running task install... 2024-04-24T23:01:54.654 DEBUG:teuthology.task.install:project ceph 2024-04-24T23:01:54.654 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T23:01:54.654 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T23:01:54.655 INFO:teuthology.task.install:Using flavor: default 2024-04-24T23:01:54.664 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T23:01:54.664 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-24T23:01:54.664 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:01:54.665 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:54.667 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:01:54.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:54.668 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:01:54.668 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:01:54.817 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:01:54.818 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:01:54.888 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:01:54.888 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:01:54.894 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-24T23:01:54.894 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-24T23:01:54.930 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:01:54.931 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:01:54.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:01:54.956 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:01:54.956 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:01:54.998 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:01:54.998 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:01:54.998 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:01:55.006 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:01:55.006 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:01:55.006 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:01:55.023 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:01:55.023 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:01:55.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:01:55.030 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-24T23:01:55.030 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:01:55.087 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:01:55.095 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:01:55.097 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:01:55.161 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-04-24T23:01:55.162 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:01:55.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-04-24T23:01:55.167 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:01:55.225 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-04-24T23:01:55.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-04-24T23:01:55.230 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-04-24T23:01:55.231 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-04-24T23:01:55.387 INFO:teuthology.orchestra.run.smithi112.stdout:48 files removed 2024-04-24T23:01:55.414 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:01:55.451 INFO:teuthology.orchestra.run.smithi150.stdout:48 files removed 2024-04-24T23:01:55.455 INFO:teuthology.orchestra.run.smithi187.stdout:48 files removed 2024-04-24T23:01:55.478 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:01:55.483 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-24T23:01:55.995 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 259 kB/s | 83 kB 00:00 2024-04-24T23:01:56.097 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 238 kB/s | 83 kB 00:00 2024-04-24T23:01:56.128 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 215 kB/s | 83 kB 00:00 2024-04-24T23:01:56.429 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-04-24T23:01:56.464 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 36 kB/s | 11 kB 00:00 2024-04-24T23:01:56.496 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-24T23:01:56.797 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-04-24T23:01:56.821 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-24T23:01:56.857 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-24T23:01:57.913 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.0 MB 00:01 2024-04-24T23:01:57.946 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.0 MB 00:01 2024-04-24T23:01:59.915 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-24T23:02:01.765 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-04-24T23:02:03.906 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 19 MB 00:04 2024-04-24T23:02:04.524 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.2 MB 00:01 2024-04-24T23:02:08.265 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 19 MB 00:05 2024-04-24T23:02:09.909 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 3.8 kB/s | 16 kB 00:04 2024-04-24T23:02:10.581 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.2 MB 00:03 2024-04-24T23:02:12.449 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-24T23:02:13.143 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 21 MB 00:03 2024-04-24T23:02:13.357 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-04-24T23:02:15.342 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 9.3 kB/s | 16 kB 00:01 2024-04-24T23:02:16.567 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 21 MB 00:03 2024-04-24T23:02:17.126 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-24T23:02:18.226 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-04-24T23:02:20.344 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:02:20.345 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:02:20.355 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:02:20.356 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:02:20.357 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:02:20.358 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:02:20.359 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:02:20.362 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:02:20.363 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:02:20.423 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-24T23:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-24T23:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-04-24T23:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-24T23:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-04-24T23:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:02:20.432 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:02:20.433 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:02:20.434 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:02:20.435 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:02:20.436 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:02:20.437 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:02:20.438 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:02:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:02:20.440 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:02:20.441 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:02:20.442 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:02:20.443 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 4 Packages 2024-04-24T23:02:20.444 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:02:20.445 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 220 M 2024-04-24T23:02:20.445 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-24T23:02:21.575 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T23:02:21.677 INFO:teuthology.orchestra.run.smithi112.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-24T23:02:22.077 INFO:teuthology.orchestra.run.smithi112.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 2.2 MB/s | 890 kB 00:00 2024-04-24T23:02:22.169 INFO:teuthology.orchestra.run.smithi112.stdout:(3/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 5.9 MB/s | 5.4 MB 00:00 2024-04-24T23:02:22.194 INFO:teuthology.orchestra.run.smithi112.stdout:(4/152): ceph-immutable-object-cache-19.0.0-244 1.2 MB/s | 145 kB 00:00 2024-04-24T23:02:22.276 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-24T23:02:22.428 INFO:teuthology.orchestra.run.smithi112.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 9.1 MB/s | 2.4 MB 00:00 2024-04-24T23:02:22.679 INFO:teuthology.orchestra.run.smithi112.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-24T23:02:22.904 INFO:teuthology.orchestra.run.smithi112.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 9.9 MB/s | 4.7 MB 00:00 2024-04-24T23:02:23.337 INFO:teuthology.orchestra.run.smithi112.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 10 MB/s | 21 MB 00:02 2024-04-24T23:02:23.387 INFO:teuthology.orchestra.run.smithi112.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 504 kB/s | 25 kB 00:00 2024-04-24T23:02:23.692 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:02:23.693 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:02:23.704 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:02:23.704 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:02:23.705 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:02:23.706 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:02:23.707 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:02:23.710 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:02:23.711 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:02:23.729 INFO:teuthology.orchestra.run.smithi112.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 12 MB/s | 10 MB 00:00 2024-04-24T23:02:23.771 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-04-24T23:02:23.798 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-24T23:02:23.798 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-04-24T23:02:23.798 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-24T23:02:23.798 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-04-24T23:02:23.798 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:02:23.798 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:02:23.798 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:02:23.799 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:02:23.800 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-04-24T23:02:23.801 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:02:23.802 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:02:23.803 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:02:23.804 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:02:23.805 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:02:23.806 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:02:23.807 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:02:23.808 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:02:23.809 INFO:teuthology.orchestra.run.smithi150.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:02:23.810 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:02:23.810 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:02:23.810 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-04-24T23:02:23.810 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:02:23.810 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:02:23.810 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-04-24T23:02:23.811 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-04-24T23:02:23.811 INFO:teuthology.orchestra.run.smithi150.stdout:Install 148 Packages 2024-04-24T23:02:23.812 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 4 Packages 2024-04-24T23:02:23.812 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:02:23.812 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 220 M 2024-04-24T23:02:23.812 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-04-24T23:02:23.813 INFO:teuthology.orchestra.run.smithi112.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.6 MB/s | 222 kB 00:00 2024-04-24T23:02:23.880 INFO:teuthology.orchestra.run.smithi112.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 406 kB/s | 27 kB 00:00 2024-04-24T23:02:23.947 INFO:teuthology.orchestra.run.smithi112.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 476 kB/s | 32 kB 00:00 2024-04-24T23:02:24.105 INFO:teuthology.orchestra.run.smithi112.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 4.6 MB/s | 745 kB 00:00 2024-04-24T23:02:24.181 INFO:teuthology.orchestra.run.smithi112.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-24T23:02:24.265 INFO:teuthology.orchestra.run.smithi112.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.5 MB/s | 127 kB 00:00 2024-04-24T23:02:24.357 INFO:teuthology.orchestra.run.smithi112.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 5.3 MB/s | 501 kB 00:00 2024-04-24T23:02:24.385 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:02:24.386 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-24T23:02:24.396 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-24T23:02:24.397 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-24T23:02:24.398 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-24T23:02:24.398 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-24T23:02:24.400 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-24T23:02:24.402 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-24T23:02:24.403 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-24T23:02:24.463 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-04-24T23:02:24.475 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-24T23:02:24.476 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-24T23:02:24.477 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-24T23:02:24.478 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-24T23:02:24.479 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-24T23:02:24.480 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:02:24.481 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-24T23:02:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:02:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:02:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-24T23:02:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-24T23:02:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-24T23:02:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-24T23:02:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:02:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-24T23:02:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:02:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:02:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:02:24.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:02:24.487 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-24T23:02:24.488 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout:Install 148 Packages 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 4 Packages 2024-04-24T23:02:24.489 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:02:24.490 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 220 M 2024-04-24T23:02:24.490 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-04-24T23:02:24.501 INFO:teuthology.orchestra.run.smithi150.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-24T23:02:25.231 INFO:teuthology.orchestra.run.smithi187.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-24T23:02:25.231 INFO:teuthology.orchestra.run.smithi112.stdout:(18/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 6.6 MB/s | 17 MB 00:02 2024-04-24T23:02:25.315 INFO:teuthology.orchestra.run.smithi112.stdout:(19/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.5 MB/s | 5.3 MB 00:00 2024-04-24T23:02:25.340 INFO:teuthology.orchestra.run.smithi112.stdout:(20/152): python3-ceph-argparse-19.0.0-2445.gd2 416 kB/s | 45 kB 00:00 2024-04-24T23:02:25.351 INFO:teuthology.orchestra.run.smithi150.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 890 kB 00:00 2024-04-24T23:02:25.366 INFO:teuthology.orchestra.run.smithi112.stdout:(21/152): python3-ceph-common-19.0.0-2445.gd2c4 2.6 MB/s | 133 kB 00:00 2024-04-24T23:02:25.400 INFO:teuthology.orchestra.run.smithi112.stdout:(22/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.7 MB/s | 162 kB 00:00 2024-04-24T23:02:25.434 INFO:teuthology.orchestra.run.smithi112.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 4.7 MB/s | 321 kB 00:00 2024-04-24T23:02:25.477 INFO:teuthology.orchestra.run.smithi150.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.1 MB/s | 145 kB 00:00 2024-04-24T23:02:25.525 INFO:teuthology.orchestra.run.smithi112.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.3 MB/s | 299 kB 00:00 2024-04-24T23:02:25.550 INFO:teuthology.orchestra.run.smithi112.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 851 kB/s | 99 kB 00:00 2024-04-24T23:02:25.634 INFO:teuthology.orchestra.run.smithi112.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 784 kB/s | 85 kB 00:00 2024-04-24T23:02:25.638 INFO:teuthology.orchestra.run.smithi187.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 2.1 MB/s | 890 kB 00:00 2024-04-24T23:02:25.751 INFO:teuthology.orchestra.run.smithi112.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.4 MB/s | 171 kB 00:00 2024-04-24T23:02:25.852 INFO:teuthology.orchestra.run.smithi112.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 318 kB/s | 32 kB 00:00 2024-04-24T23:02:25.960 INFO:teuthology.orchestra.run.smithi112.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.3 MB/s | 141 kB 00:00 2024-04-24T23:02:26.219 INFO:teuthology.orchestra.run.smithi187.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 248 kB/s | 145 kB 00:00 2024-04-24T23:02:26.235 INFO:teuthology.orchestra.run.smithi112.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.5 MB/s | 3.1 MB 00:00 2024-04-24T23:02:26.327 INFO:teuthology.orchestra.run.smithi150.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-24T23:02:26.578 INFO:teuthology.orchestra.run.smithi112.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 5.7 MB/s | 3.5 MB 00:00 2024-04-24T23:02:26.670 INFO:teuthology.orchestra.run.smithi112.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.7 MB/s | 249 kB 00:00 2024-04-24T23:02:26.686 INFO:teuthology.orchestra.run.smithi150.stdout:(5/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 2.4 MB 00:01 2024-04-24T23:02:26.753 INFO:teuthology.orchestra.run.smithi112.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 593 kB/s | 49 kB 00:00 2024-04-24T23:02:26.845 INFO:teuthology.orchestra.run.smithi112.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 167 kB/s | 15 kB 00:00 2024-04-24T23:02:26.919 INFO:teuthology.orchestra.run.smithi150.stdout:(6/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-04-24T23:02:26.937 INFO:teuthology.orchestra.run.smithi112.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.8 MB/s | 264 kB 00:00 2024-04-24T23:02:27.121 INFO:teuthology.orchestra.run.smithi112.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 4.1 MB/s | 762 kB 00:00 2024-04-24T23:02:27.203 INFO:teuthology.orchestra.run.smithi187.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-04-24T23:02:27.213 INFO:teuthology.orchestra.run.smithi112.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 199 kB/s | 18 kB 00:00 2024-04-24T23:02:27.561 INFO:teuthology.orchestra.run.smithi187.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 4.2 MB/s | 1.5 MB 00:00 2024-04-24T23:02:27.655 INFO:teuthology.orchestra.run.smithi112.stdout:(38/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 11 MB/s | 48 MB 00:04 2024-04-24T23:02:27.755 INFO:teuthology.orchestra.run.smithi112.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 148 kB/s | 80 kB 00:00 2024-04-24T23:02:27.839 INFO:teuthology.orchestra.run.smithi112.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 4.6 MB/s | 7.4 MB 00:01 2024-04-24T23:02:27.947 INFO:teuthology.orchestra.run.smithi112.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-24T23:02:27.986 INFO:teuthology.orchestra.run.smithi150.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 3.6 MB/s | 4.7 MB 00:01 2024-04-24T23:02:27.997 INFO:teuthology.orchestra.run.smithi112.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-04-24T23:02:28.048 INFO:teuthology.orchestra.run.smithi112.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-04-24T23:02:28.140 INFO:teuthology.orchestra.run.smithi112.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-24T23:02:28.165 INFO:teuthology.orchestra.run.smithi112.stdout:(45/152): python3-ply-3.11-14.el9.noarch.rpm 637 kB/s | 106 kB 00:00 2024-04-24T23:02:28.203 INFO:teuthology.orchestra.run.smithi187.stdout:(6/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-24T23:02:28.257 INFO:teuthology.orchestra.run.smithi112.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-24T23:02:28.282 INFO:teuthology.orchestra.run.smithi112.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-24T23:02:28.412 INFO:teuthology.orchestra.run.smithi187.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 5.5 MB/s | 4.7 MB 00:00 2024-04-24T23:02:28.549 INFO:teuthology.orchestra.run.smithi112.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-24T23:02:28.574 INFO:teuthology.orchestra.run.smithi112.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-04-24T23:02:28.608 INFO:teuthology.orchestra.run.smithi112.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-24T23:02:28.625 INFO:teuthology.orchestra.run.smithi112.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-04-24T23:02:28.675 INFO:teuthology.orchestra.run.smithi112.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 658 kB/s | 33 kB 00:00 2024-04-24T23:02:28.700 INFO:teuthology.orchestra.run.smithi112.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-04-24T23:02:28.776 INFO:teuthology.orchestra.run.smithi112.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 243 kB/s | 18 kB 00:00 2024-04-24T23:02:28.843 INFO:teuthology.orchestra.run.smithi112.stdout:(55/152): python3-cryptography-36.0.1-4.el9.x86 1.4 MB/s | 1.2 MB 00:00 2024-04-24T23:02:28.868 INFO:teuthology.orchestra.run.smithi112.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-04-24T23:02:28.960 INFO:teuthology.orchestra.run.smithi112.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 494 kB/s | 45 kB 00:00 2024-04-24T23:02:29.077 INFO:teuthology.orchestra.run.smithi112.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-04-24T23:02:29.144 INFO:teuthology.orchestra.run.smithi112.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-24T23:02:29.177 INFO:teuthology.orchestra.run.smithi112.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.0 MB/s | 662 kB 00:00 2024-04-24T23:02:29.211 INFO:teuthology.orchestra.run.smithi112.stdout:(61/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 435 kB/s | 160 kB 00:00 2024-04-24T23:02:29.303 INFO:teuthology.orchestra.run.smithi112.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 160 kB/s | 25 kB 00:00 2024-04-24T23:02:29.329 INFO:teuthology.orchestra.run.smithi112.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 315 kB/s | 47 kB 00:00 2024-04-24T23:02:29.387 INFO:teuthology.orchestra.run.smithi112.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-24T23:02:29.412 INFO:teuthology.orchestra.run.smithi112.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-04-24T23:02:29.454 INFO:teuthology.orchestra.run.smithi112.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-04-24T23:02:29.522 INFO:teuthology.orchestra.run.smithi150.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 6.5 MB/s | 17 MB 00:02 2024-04-24T23:02:29.569 INFO:teuthology.orchestra.run.smithi150.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 505 kB/s | 25 kB 00:00 2024-04-24T23:02:29.680 INFO:teuthology.orchestra.run.smithi112.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-04-24T23:02:29.772 INFO:teuthology.orchestra.run.smithi112.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-04-24T23:02:29.778 INFO:teuthology.orchestra.run.smithi150.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 5.7 MB/s | 10 MB 00:01 2024-04-24T23:02:29.830 INFO:teuthology.orchestra.run.smithi112.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T23:02:29.853 INFO:teuthology.orchestra.run.smithi150.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.9 MB/s | 222 kB 00:00 2024-04-24T23:02:29.888 INFO:teuthology.orchestra.run.smithi112.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-24T23:02:29.904 INFO:teuthology.orchestra.run.smithi150.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 542 kB/s | 27 kB 00:00 2024-04-24T23:02:29.954 INFO:teuthology.orchestra.run.smithi150.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 634 kB/s | 32 kB 00:00 2024-04-24T23:02:30.006 INFO:teuthology.orchestra.run.smithi112.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-04-24T23:02:30.073 INFO:teuthology.orchestra.run.smithi112.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-24T23:02:30.087 INFO:teuthology.orchestra.run.smithi150.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 5.5 MB/s | 745 kB 00:00 2024-04-24T23:02:30.112 INFO:teuthology.orchestra.run.smithi187.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 6.0 MB/s | 10 MB 00:01 2024-04-24T23:02:30.138 INFO:teuthology.orchestra.run.smithi150.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 3.2 MB/s | 163 kB 00:00 2024-04-24T23:02:30.162 INFO:teuthology.orchestra.run.smithi187.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 506 kB/s | 25 kB 00:00 2024-04-24T23:02:30.173 INFO:teuthology.orchestra.run.smithi112.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-04-24T23:02:30.188 INFO:teuthology.orchestra.run.smithi150.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 2.5 MB/s | 127 kB 00:00 2024-04-24T23:02:30.206 INFO:teuthology.orchestra.run.smithi112.stdout:(74/152): postgresql-13.14-1.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-24T23:02:30.232 INFO:teuthology.orchestra.run.smithi112.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T23:02:30.290 INFO:teuthology.orchestra.run.smithi112.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-04-24T23:02:30.313 INFO:teuthology.orchestra.run.smithi150.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.9 MB/s | 501 kB 00:00 2024-04-24T23:02:30.316 INFO:teuthology.orchestra.run.smithi112.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-24T23:02:30.349 INFO:teuthology.orchestra.run.smithi112.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-04-24T23:02:30.408 INFO:teuthology.orchestra.run.smithi112.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-24T23:02:30.433 INFO:teuthology.orchestra.run.smithi112.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 416 kB/s | 35 kB 00:00 2024-04-24T23:02:30.500 INFO:teuthology.orchestra.run.smithi112.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-24T23:02:30.559 INFO:teuthology.orchestra.run.smithi112.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-24T23:02:30.634 INFO:teuthology.orchestra.run.smithi112.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-24T23:02:30.659 INFO:teuthology.orchestra.run.smithi112.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 770 kB/s | 77 kB 00:00 2024-04-24T23:02:30.685 INFO:teuthology.orchestra.run.smithi112.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-04-24T23:02:30.718 INFO:teuthology.orchestra.run.smithi112.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-24T23:02:30.744 INFO:teuthology.orchestra.run.smithi112.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-24T23:02:30.777 INFO:teuthology.orchestra.run.smithi112.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-24T23:02:30.794 INFO:teuthology.orchestra.run.smithi112.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T23:02:30.828 INFO:teuthology.orchestra.run.smithi112.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T23:02:30.886 INFO:teuthology.orchestra.run.smithi112.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-04-24T23:02:30.946 INFO:teuthology.orchestra.run.smithi112.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-04-24T23:02:31.012 INFO:teuthology.orchestra.run.smithi112.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-24T23:02:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-04-24T23:02:31.129 INFO:teuthology.orchestra.run.smithi112.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-04-24T23:02:31.196 INFO:teuthology.orchestra.run.smithi112.stdout:(96/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.5 MB/s | 5.1 MB 00:00 2024-04-24T23:02:31.221 INFO:teuthology.orchestra.run.smithi112.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-24T23:02:31.263 INFO:teuthology.orchestra.run.smithi112.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 866 kB/s | 58 kB 00:00 2024-04-24T23:02:31.279 INFO:teuthology.orchestra.run.smithi187.stdout:(10/152): ceph-common-19.0.0-2445.gd2c45d97.el9 3.3 MB/s | 21 MB 00:06 2024-04-24T23:02:31.288 INFO:teuthology.orchestra.run.smithi112.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-24T23:02:31.322 INFO:teuthology.orchestra.run.smithi150.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.3 MB/s | 5.3 MB 00:01 2024-04-24T23:02:31.322 INFO:teuthology.orchestra.run.smithi112.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-24T23:02:31.350 INFO:teuthology.orchestra.run.smithi112.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 738 kB/s | 45 kB 00:00 2024-04-24T23:02:31.364 INFO:teuthology.orchestra.run.smithi150.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 1.1 MB/s | 45 kB 00:00 2024-04-24T23:02:31.379 INFO:teuthology.orchestra.run.smithi187.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.2 MB/s | 222 kB 00:00 2024-04-24T23:02:31.383 INFO:teuthology.orchestra.run.smithi112.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 369 kB/s | 22 kB 00:00 2024-04-24T23:02:31.414 INFO:teuthology.orchestra.run.smithi150.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.6 MB/s | 133 kB 00:00 2024-04-24T23:02:31.425 INFO:teuthology.orchestra.run.smithi112.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-24T23:02:31.454 INFO:teuthology.orchestra.run.smithi187.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 362 kB/s | 27 kB 00:00 2024-04-24T23:02:31.464 INFO:teuthology.orchestra.run.smithi150.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 3.2 MB/s | 162 kB 00:00 2024-04-24T23:02:31.538 INFO:teuthology.orchestra.run.smithi187.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 381 kB/s | 32 kB 00:00 2024-04-24T23:02:31.549 INFO:teuthology.orchestra.run.smithi150.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.7 MB/s | 321 kB 00:00 2024-04-24T23:02:31.625 INFO:teuthology.orchestra.run.smithi112.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-24T23:02:31.649 INFO:teuthology.orchestra.run.smithi150.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.9 MB/s | 299 kB 00:00 2024-04-24T23:02:31.675 INFO:teuthology.orchestra.run.smithi112.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 392 kB/s | 19 kB 00:00 2024-04-24T23:02:31.699 INFO:teuthology.orchestra.run.smithi150.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 2.0 MB/s | 99 kB 00:00 2024-04-24T23:02:31.701 INFO:teuthology.orchestra.run.smithi112.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-24T23:02:31.726 INFO:teuthology.orchestra.run.smithi112.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-24T23:02:31.746 INFO:teuthology.orchestra.run.smithi187.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.5 MB/s | 745 kB 00:00 2024-04-24T23:02:31.749 INFO:teuthology.orchestra.run.smithi150.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-24T23:02:31.751 INFO:teuthology.orchestra.run.smithi112.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-04-24T23:02:31.802 INFO:teuthology.orchestra.run.smithi112.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 647 kB/s | 49 kB 00:00 2024-04-24T23:02:31.813 INFO:teuthology.orchestra.run.smithi187.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 163 kB 00:00 2024-04-24T23:02:31.827 INFO:teuthology.orchestra.run.smithi112.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-04-24T23:02:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-24T23:02:31.888 INFO:teuthology.orchestra.run.smithi187.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.7 MB/s | 127 kB 00:00 2024-04-24T23:02:31.928 INFO:teuthology.orchestra.run.smithi112.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-24T23:02:31.969 INFO:teuthology.orchestra.run.smithi112.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.8 MB/s | 838 kB 00:00 2024-04-24T23:02:31.995 INFO:teuthology.orchestra.run.smithi112.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 651 kB/s | 43 kB 00:00 2024-04-24T23:02:32.020 INFO:teuthology.orchestra.run.smithi112.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-04-24T23:02:32.045 INFO:teuthology.orchestra.run.smithi112.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-24T23:02:32.070 INFO:teuthology.orchestra.run.smithi112.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-24T23:02:32.072 INFO:teuthology.orchestra.run.smithi187.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 2.7 MB/s | 501 kB 00:00 2024-04-24T23:02:32.096 INFO:teuthology.orchestra.run.smithi112.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-24T23:02:32.129 INFO:teuthology.orchestra.run.smithi112.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-24T23:02:32.154 INFO:teuthology.orchestra.run.smithi112.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 605 kB/s | 35 kB 00:00 2024-04-24T23:02:32.180 INFO:teuthology.orchestra.run.smithi112.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-24T23:02:32.205 INFO:teuthology.orchestra.run.smithi112.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-24T23:02:32.230 INFO:teuthology.orchestra.run.smithi112.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-24T23:02:32.255 INFO:teuthology.orchestra.run.smithi112.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-24T23:02:32.281 INFO:teuthology.orchestra.run.smithi112.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-24T23:02:32.298 INFO:teuthology.orchestra.run.smithi112.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 552 kB/s | 23 kB 00:00 2024-04-24T23:02:32.315 INFO:teuthology.orchestra.run.smithi112.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 578 kB/s | 19 kB 00:00 2024-04-24T23:02:32.333 INFO:teuthology.orchestra.run.smithi150.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 5.3 MB/s | 3.1 MB 00:00 2024-04-24T23:02:32.340 INFO:teuthology.orchestra.run.smithi112.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 470 kB/s | 20 kB 00:00 2024-04-24T23:02:32.383 INFO:teuthology.orchestra.run.smithi150.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 3.3 MB/s | 171 kB 00:00 2024-04-24T23:02:32.390 INFO:teuthology.orchestra.run.smithi112.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-04-24T23:02:32.407 INFO:teuthology.orchestra.run.smithi112.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 690 kB/s | 46 kB 00:00 2024-04-24T23:02:32.441 INFO:teuthology.orchestra.run.smithi112.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-24T23:02:32.442 INFO:teuthology.orchestra.run.smithi150.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 545 kB/s | 32 kB 00:00 2024-04-24T23:02:32.467 INFO:teuthology.orchestra.run.smithi112.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-04-24T23:02:32.492 INFO:teuthology.orchestra.run.smithi112.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-04-24T23:02:32.500 INFO:teuthology.orchestra.run.smithi150.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.4 MB/s | 141 kB 00:00 2024-04-24T23:02:32.509 INFO:teuthology.orchestra.run.smithi112.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 393 kB/s | 16 kB 00:00 2024-04-24T23:02:32.534 INFO:teuthology.orchestra.run.smithi112.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-04-24T23:02:32.551 INFO:teuthology.orchestra.run.smithi112.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 674 kB/s | 28 kB 00:00 2024-04-24T23:02:32.576 INFO:teuthology.orchestra.run.smithi112.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 736 kB/s | 31 kB 00:00 2024-04-24T23:02:32.610 INFO:teuthology.orchestra.run.smithi112.stdout:(138/152): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-04-24T23:02:32.635 INFO:teuthology.orchestra.run.smithi112.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-24T23:02:32.660 INFO:teuthology.orchestra.run.smithi112.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-04-24T23:02:32.686 INFO:teuthology.orchestra.run.smithi112.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-24T23:02:32.711 INFO:teuthology.orchestra.run.smithi112.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-04-24T23:02:32.736 INFO:teuthology.orchestra.run.smithi112.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-24T23:02:32.761 INFO:teuthology.orchestra.run.smithi112.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-24T23:02:32.787 INFO:teuthology.orchestra.run.smithi112.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-04-24T23:02:32.812 INFO:teuthology.orchestra.run.smithi112.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-24T23:02:32.837 INFO:teuthology.orchestra.run.smithi112.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-24T23:02:32.879 INFO:teuthology.orchestra.run.smithi112.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-24T23:02:33.204 INFO:teuthology.orchestra.run.smithi112.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.2 MB/s | 1.3 MB 00:00 2024-04-24T23:02:33.276 INFO:teuthology.orchestra.run.smithi150.stdout:(30/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 4.5 MB/s | 3.5 MB 00:00 2024-04-24T23:02:33.522 INFO:teuthology.orchestra.run.smithi187.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 3.7 MB/s | 5.3 MB 00:01 2024-04-24T23:02:33.622 INFO:teuthology.orchestra.run.smithi187.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 453 kB/s | 45 kB 00:00 2024-04-24T23:02:33.671 INFO:teuthology.orchestra.run.smithi112.stdout:(150/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.9 MB/s | 3.4 MB 00:00 2024-04-24T23:02:33.698 INFO:teuthology.orchestra.run.smithi187.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.7 MB/s | 133 kB 00:00 2024-04-24T23:02:33.722 INFO:teuthology.orchestra.run.smithi112.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-04-24T23:02:33.782 INFO:teuthology.orchestra.run.smithi187.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.9 MB/s | 162 kB 00:00 2024-04-24T23:02:33.924 INFO:teuthology.orchestra.run.smithi187.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.2 MB/s | 321 kB 00:00 2024-04-24T23:02:34.132 INFO:teuthology.orchestra.run.smithi187.stdout:(23/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 2.8 MB/s | 17 MB 00:05 2024-04-24T23:02:34.158 INFO:teuthology.orchestra.run.smithi187.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.3 MB/s | 299 kB 00:00 2024-04-24T23:02:34.208 INFO:teuthology.orchestra.run.smithi187.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.3 MB/s | 99 kB 00:00 2024-04-24T23:02:34.233 INFO:teuthology.orchestra.run.smithi187.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-24T23:02:34.305 INFO:teuthology.orchestra.run.smithi112.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-04-24T23:02:34.308 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:34.308 INFO:teuthology.orchestra.run.smithi112.stdout:Total 16 MB/s | 220 MB 00:13 2024-04-24T23:02:34.317 INFO:teuthology.orchestra.run.smithi187.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-24T23:02:34.392 INFO:teuthology.orchestra.run.smithi187.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 424 kB/s | 32 kB 00:00 2024-04-24T23:02:34.476 INFO:teuthology.orchestra.run.smithi187.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.7 MB/s | 141 kB 00:00 2024-04-24T23:02:34.692 INFO:teuthology.orchestra.run.smithi150.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-04-24T23:02:34.759 INFO:teuthology.orchestra.run.smithi150.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.7 MB/s | 249 kB 00:00 2024-04-24T23:02:34.818 INFO:teuthology.orchestra.run.smithi150.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 843 kB/s | 49 kB 00:00 2024-04-24T23:02:34.877 INFO:teuthology.orchestra.run.smithi150.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 263 kB/s | 15 kB 00:00 2024-04-24T23:02:34.994 INFO:teuthology.orchestra.run.smithi150.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 264 kB 00:00 2024-04-24T23:02:35.101 INFO:teuthology.orchestra.run.smithi187.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 3.5 MB/s | 3.1 MB 00:00 2024-04-24T23:02:35.144 INFO:teuthology.orchestra.run.smithi150.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-24T23:02:35.202 INFO:teuthology.orchestra.run.smithi150.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 314 kB/s | 18 kB 00:00 2024-04-24T23:02:35.301 INFO:teuthology.orchestra.run.smithi187.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 4.3 MB/s | 3.5 MB 00:00 2024-04-24T23:02:35.402 INFO:teuthology.orchestra.run.smithi187.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.4 MB/s | 249 kB 00:00 2024-04-24T23:02:35.419 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-24T23:02:35.460 INFO:teuthology.orchestra.run.smithi187.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 847 kB/s | 49 kB 00:00 2024-04-24T23:02:35.520 INFO:teuthology.orchestra.run.smithi150.stdout:(38/152): ceph-common-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 21 MB 00:11 2024-04-24T23:02:35.527 INFO:teuthology.orchestra.run.smithi187.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 230 kB/s | 15 kB 00:00 2024-04-24T23:02:35.563 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-24T23:02:35.563 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-24T23:02:35.627 INFO:teuthology.orchestra.run.smithi187.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.6 MB/s | 264 kB 00:00 2024-04-24T23:02:35.636 INFO:teuthology.orchestra.run.smithi150.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 185 kB/s | 80 kB 00:00 2024-04-24T23:02:35.844 INFO:teuthology.orchestra.run.smithi187.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.4 MB/s | 762 kB 00:00 2024-04-24T23:02:35.911 INFO:teuthology.orchestra.run.smithi187.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 274 kB/s | 18 kB 00:00 2024-04-24T23:02:36.053 INFO:teuthology.orchestra.run.smithi187.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-24T23:02:36.178 INFO:teuthology.orchestra.run.smithi187.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-24T23:02:36.212 INFO:teuthology.orchestra.run.smithi150.stdout:(40/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.3 MB/s | 48 MB 00:06 2024-04-24T23:02:36.229 INFO:teuthology.orchestra.run.smithi187.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-24T23:02:36.237 INFO:teuthology.orchestra.run.smithi150.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 325 kB/s | 195 kB 00:00 2024-04-24T23:02:36.279 INFO:teuthology.orchestra.run.smithi187.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-24T23:02:36.371 INFO:teuthology.orchestra.run.smithi187.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-24T23:02:36.413 INFO:teuthology.orchestra.run.smithi187.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-24T23:02:36.463 INFO:teuthology.orchestra.run.smithi187.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-24T23:02:36.496 INFO:teuthology.orchestra.run.smithi150.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 823 kB/s | 803 kB 00:00 2024-04-24T23:02:36.513 INFO:teuthology.orchestra.run.smithi187.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-24T23:02:36.564 INFO:teuthology.orchestra.run.smithi187.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-24T23:02:36.604 INFO:teuthology.orchestra.run.smithi150.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 985 kB/s | 106 kB 00:00 2024-04-24T23:02:36.713 INFO:teuthology.orchestra.run.smithi150.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-24T23:02:36.763 INFO:teuthology.orchestra.run.smithi150.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-04-24T23:02:36.788 INFO:teuthology.orchestra.run.smithi150.stdout:(46/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 439 kB/s | 253 kB 00:00 2024-04-24T23:02:36.813 INFO:teuthology.orchestra.run.smithi150.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-24T23:02:36.872 INFO:teuthology.orchestra.run.smithi150.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-24T23:02:37.023 INFO:teuthology.orchestra.run.smithi187.stdout:(47/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 7.0 MB/s | 48 MB 00:06 2024-04-24T23:02:37.031 INFO:teuthology.orchestra.run.smithi150.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-04-24T23:02:37.056 INFO:teuthology.orchestra.run.smithi150.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 583 kB/s | 107 kB 00:00 2024-04-24T23:02:37.090 INFO:teuthology.orchestra.run.smithi150.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-04-24T23:02:37.114 INFO:teuthology.orchestra.run.smithi187.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 3.7 MB/s | 7.4 MB 00:02 2024-04-24T23:02:37.115 INFO:teuthology.orchestra.run.smithi150.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 256 kB/s | 21 kB 00:00 2024-04-24T23:02:37.148 INFO:teuthology.orchestra.run.smithi150.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-04-24T23:02:37.173 INFO:teuthology.orchestra.run.smithi187.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-04-24T23:02:37.174 INFO:teuthology.orchestra.run.smithi150.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-04-24T23:02:37.206 INFO:teuthology.orchestra.run.smithi187.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 650 kB/s | 21 kB 00:00 2024-04-24T23:02:37.207 INFO:teuthology.orchestra.run.smithi150.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-04-24T23:02:37.233 INFO:teuthology.orchestra.run.smithi150.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-24T23:02:37.240 INFO:teuthology.orchestra.run.smithi187.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 853 kB/s | 107 kB 00:00 2024-04-24T23:02:37.265 INFO:teuthology.orchestra.run.smithi187.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-24T23:02:37.266 INFO:teuthology.orchestra.run.smithi150.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-24T23:02:37.291 INFO:teuthology.orchestra.run.smithi187.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-04-24T23:02:37.292 INFO:teuthology.orchestra.run.smithi150.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-24T23:02:37.309 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-24T23:02:37.309 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-24T23:02:37.324 INFO:teuthology.orchestra.run.smithi187.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-04-24T23:02:37.342 INFO:teuthology.orchestra.run.smithi150.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-24T23:02:37.349 INFO:teuthology.orchestra.run.smithi187.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-24T23:02:37.374 INFO:teuthology.orchestra.run.smithi187.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-24T23:02:37.375 INFO:teuthology.orchestra.run.smithi150.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-24T23:02:37.400 INFO:teuthology.orchestra.run.smithi187.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-24T23:02:37.401 INFO:teuthology.orchestra.run.smithi150.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-24T23:02:37.433 INFO:teuthology.orchestra.run.smithi187.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-24T23:02:37.434 INFO:teuthology.orchestra.run.smithi150.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-04-24T23:02:37.460 INFO:teuthology.orchestra.run.smithi150.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-04-24T23:02:37.467 INFO:teuthology.orchestra.run.smithi187.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.6 MB/s | 662 kB 00:00 2024-04-24T23:02:37.485 INFO:teuthology.orchestra.run.smithi150.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-24T23:02:37.510 INFO:teuthology.orchestra.run.smithi150.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-24T23:02:37.526 INFO:teuthology.orchestra.run.smithi187.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-24T23:02:37.535 INFO:teuthology.orchestra.run.smithi150.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-04-24T23:02:37.551 INFO:teuthology.orchestra.run.smithi187.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-04-24T23:02:37.576 INFO:teuthology.orchestra.run.smithi187.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-24T23:02:37.602 INFO:teuthology.orchestra.run.smithi187.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-24T23:02:37.627 INFO:teuthology.orchestra.run.smithi187.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-24T23:02:37.652 INFO:teuthology.orchestra.run.smithi187.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-04-24T23:02:37.653 INFO:teuthology.orchestra.run.smithi150.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-24T23:02:37.677 INFO:teuthology.orchestra.run.smithi187.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-24T23:02:37.719 INFO:teuthology.orchestra.run.smithi150.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-24T23:02:37.753 INFO:teuthology.orchestra.run.smithi150.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-24T23:02:37.769 INFO:teuthology.orchestra.run.smithi187.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-24T23:02:37.770 INFO:teuthology.orchestra.run.smithi150.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-24T23:02:37.803 INFO:teuthology.orchestra.run.smithi187.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-24T23:02:37.828 INFO:teuthology.orchestra.run.smithi150.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-24T23:02:37.837 INFO:teuthology.orchestra.run.smithi187.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-24T23:02:37.862 INFO:teuthology.orchestra.run.smithi187.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-24T23:02:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-24T23:02:37.895 INFO:teuthology.orchestra.run.smithi150.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-24T23:02:37.954 INFO:teuthology.orchestra.run.smithi150.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-24T23:02:38.020 INFO:teuthology.orchestra.run.smithi187.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-24T23:02:38.021 INFO:teuthology.orchestra.run.smithi150.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-24T23:02:38.054 INFO:teuthology.orchestra.run.smithi187.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-24T23:02:38.079 INFO:teuthology.orchestra.run.smithi187.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-24T23:02:38.096 INFO:teuthology.orchestra.run.smithi150.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-04-24T23:02:38.121 INFO:teuthology.orchestra.run.smithi187.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-24T23:02:38.122 INFO:teuthology.orchestra.run.smithi150.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-04-24T23:02:38.188 INFO:teuthology.orchestra.run.smithi187.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-24T23:02:38.197 INFO:teuthology.orchestra.run.smithi150.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-24T23:02:38.213 INFO:teuthology.orchestra.run.smithi187.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-24T23:02:38.231 INFO:teuthology.orchestra.run.smithi150.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-04-24T23:02:38.238 INFO:teuthology.orchestra.run.smithi187.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-24T23:02:38.248 INFO:teuthology.orchestra.run.smithi150.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-04-24T23:02:38.256 INFO:teuthology.orchestra.run.smithi187.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-04-24T23:02:38.273 INFO:teuthology.orchestra.run.smithi150.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-24T23:02:38.281 INFO:teuthology.orchestra.run.smithi187.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-24T23:02:38.298 INFO:teuthology.orchestra.run.smithi187.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 829 kB/s | 35 kB 00:00 2024-04-24T23:02:38.306 INFO:teuthology.orchestra.run.smithi150.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-24T23:02:38.340 INFO:teuthology.orchestra.run.smithi150.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-24T23:02:38.390 INFO:teuthology.orchestra.run.smithi187.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-24T23:02:38.390 INFO:teuthology.orchestra.run.smithi150.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-24T23:02:38.407 INFO:teuthology.orchestra.run.smithi150.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 769 kB/s | 77 kB 00:00 2024-04-24T23:02:38.415 INFO:teuthology.orchestra.run.smithi187.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-04-24T23:02:38.440 INFO:teuthology.orchestra.run.smithi187.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-24T23:02:38.466 INFO:teuthology.orchestra.run.smithi187.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-24T23:02:38.474 INFO:teuthology.orchestra.run.smithi150.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-24T23:02:38.482 INFO:teuthology.orchestra.run.smithi187.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-24T23:02:38.499 INFO:teuthology.orchestra.run.smithi150.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-24T23:02:38.508 INFO:teuthology.orchestra.run.smithi187.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-04-24T23:02:38.525 INFO:teuthology.orchestra.run.smithi150.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-04-24T23:02:38.533 INFO:teuthology.orchestra.run.smithi187.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-24T23:02:38.550 INFO:teuthology.orchestra.run.smithi150.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-24T23:02:38.558 INFO:teuthology.orchestra.run.smithi187.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-24T23:02:38.575 INFO:teuthology.orchestra.run.smithi150.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-04-24T23:02:38.584 INFO:teuthology.orchestra.run.smithi187.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-24T23:02:38.601 INFO:teuthology.orchestra.run.smithi150.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-04-24T23:02:38.609 INFO:teuthology.orchestra.run.smithi187.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-24T23:02:38.634 INFO:teuthology.orchestra.run.smithi150.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 711 kB/s | 42 kB 00:00 2024-04-24T23:02:38.660 INFO:teuthology.orchestra.run.smithi150.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-04-24T23:02:38.693 INFO:teuthology.orchestra.run.smithi187.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-24T23:02:38.693 INFO:teuthology.orchestra.run.smithi150.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-24T23:02:38.718 INFO:teuthology.orchestra.run.smithi187.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-04-24T23:02:38.718 INFO:teuthology.orchestra.run.smithi150.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-04-24T23:02:38.743 INFO:teuthology.orchestra.run.smithi187.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-04-24T23:02:38.752 INFO:teuthology.orchestra.run.smithi150.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-04-24T23:02:38.768 INFO:teuthology.orchestra.run.smithi187.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-24T23:02:38.785 INFO:teuthology.orchestra.run.smithi150.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-24T23:02:38.793 INFO:teuthology.orchestra.run.smithi187.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-04-24T23:02:38.802 INFO:teuthology.orchestra.run.smithi150.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T23:02:38.819 INFO:teuthology.orchestra.run.smithi187.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-04-24T23:02:38.844 INFO:teuthology.orchestra.run.smithi187.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-24T23:02:38.853 INFO:teuthology.orchestra.run.smithi150.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-24T23:02:38.869 INFO:teuthology.orchestra.run.smithi187.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T23:02:38.870 INFO:teuthology.orchestra.run.smithi150.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 950 kB/s | 64 kB 00:00 2024-04-24T23:02:38.895 INFO:teuthology.orchestra.run.smithi187.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-24T23:02:38.903 INFO:teuthology.orchestra.run.smithi150.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-04-24T23:02:38.920 INFO:teuthology.orchestra.run.smithi187.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-24T23:02:38.937 INFO:teuthology.orchestra.run.smithi150.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-04-24T23:02:38.945 INFO:teuthology.orchestra.run.smithi187.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-04-24T23:02:38.970 INFO:teuthology.orchestra.run.smithi187.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-24T23:02:39.013 INFO:teuthology.orchestra.run.smithi187.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-24T23:02:39.096 INFO:teuthology.orchestra.run.smithi150.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-24T23:02:39.221 INFO:teuthology.orchestra.run.smithi150.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-04-24T23:02:39.280 INFO:teuthology.orchestra.run.smithi187.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-24T23:02:39.463 INFO:teuthology.orchestra.run.smithi150.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-04-24T23:02:39.530 INFO:teuthology.orchestra.run.smithi187.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-04-24T23:02:39.555 INFO:teuthology.orchestra.run.smithi187.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 570 kB/s | 309 kB 00:00 2024-04-24T23:02:39.580 INFO:teuthology.orchestra.run.smithi187.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 900 kB/s | 548 kB 00:00 2024-04-24T23:02:39.622 INFO:teuthology.orchestra.run.smithi187.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-04-24T23:02:39.638 INFO:teuthology.orchestra.run.smithi150.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 570 kB/s | 309 kB 00:00 2024-04-24T23:02:39.655 INFO:teuthology.orchestra.run.smithi187.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-04-24T23:02:39.689 INFO:teuthology.orchestra.run.smithi187.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-24T23:02:39.697 INFO:teuthology.orchestra.run.smithi150.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 722 kB/s | 548 kB 00:00 2024-04-24T23:02:39.731 INFO:teuthology.orchestra.run.smithi187.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-24T23:02:39.755 INFO:teuthology.orchestra.run.smithi150.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-04-24T23:02:39.805 INFO:teuthology.orchestra.run.smithi150.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-24T23:02:39.848 INFO:teuthology.orchestra.run.smithi187.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-24T23:02:39.866 INFO:teuthology.orchestra.run.smithi150.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 608 kB/s | 67 kB 00:00 2024-04-24T23:02:39.890 INFO:teuthology.orchestra.run.smithi187.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-04-24T23:02:39.924 INFO:teuthology.orchestra.run.smithi187.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 579 kB/s | 43 kB 00:00 2024-04-24T23:02:39.957 INFO:teuthology.orchestra.run.smithi187.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-04-24T23:02:40.000 INFO:teuthology.orchestra.run.smithi150.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 779 kB/s | 151 kB 00:00 2024-04-24T23:02:40.032 INFO:teuthology.orchestra.run.smithi187.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-04-24T23:02:40.058 INFO:teuthology.orchestra.run.smithi187.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-04-24T23:02:40.083 INFO:teuthology.orchestra.run.smithi187.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-24T23:02:40.117 INFO:teuthology.orchestra.run.smithi187.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-24T23:02:40.144 INFO:teuthology.orchestra.run.smithi187.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.4 MB/s | 303 kB 00:00 2024-04-24T23:02:40.161 INFO:teuthology.orchestra.run.smithi187.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 456 kB/s | 35 kB 00:00 2024-04-24T23:02:40.195 INFO:teuthology.orchestra.run.smithi187.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.6 MB/s | 206 kB 00:00 2024-04-24T23:02:40.236 INFO:teuthology.orchestra.run.smithi187.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-24T23:02:40.262 INFO:teuthology.orchestra.run.smithi187.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-24T23:02:40.304 INFO:teuthology.orchestra.run.smithi187.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-24T23:02:40.329 INFO:teuthology.orchestra.run.smithi187.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 191 kB/s | 18 kB 00:00 2024-04-24T23:02:40.354 INFO:teuthology.orchestra.run.smithi187.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-24T23:02:40.379 INFO:teuthology.orchestra.run.smithi187.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-24T23:02:40.405 INFO:teuthology.orchestra.run.smithi187.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-04-24T23:02:40.425 INFO:teuthology.orchestra.run.smithi150.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 1.3 MB/s | 548 kB 00:00 2024-04-24T23:02:40.455 INFO:teuthology.orchestra.run.smithi187.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-04-24T23:02:40.480 INFO:teuthology.orchestra.run.smithi187.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-04-24T23:02:40.514 INFO:teuthology.orchestra.run.smithi187.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-04-24T23:02:40.534 INFO:teuthology.orchestra.run.smithi150.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-24T23:02:40.548 INFO:teuthology.orchestra.run.smithi187.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-04-24T23:02:40.573 INFO:teuthology.orchestra.run.smithi187.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-24T23:02:40.598 INFO:teuthology.orchestra.run.smithi187.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-24T23:02:40.632 INFO:teuthology.orchestra.run.smithi187.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-24T23:02:40.634 INFO:teuthology.orchestra.run.smithi150.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-04-24T23:02:40.657 INFO:teuthology.orchestra.run.smithi187.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-04-24T23:02:40.682 INFO:teuthology.orchestra.run.smithi187.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 366 kB/s | 31 kB 00:00 2024-04-24T23:02:40.707 INFO:teuthology.orchestra.run.smithi187.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-24T23:02:40.733 INFO:teuthology.orchestra.run.smithi187.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 782 kB/s | 59 kB 00:00 2024-04-24T23:02:40.734 INFO:teuthology.orchestra.run.smithi150.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-24T23:02:40.758 INFO:teuthology.orchestra.run.smithi187.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-24T23:02:40.784 INFO:teuthology.orchestra.run.smithi150.stdout:(116/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 913 kB/s | 838 kB 00:00 2024-04-24T23:02:40.808 INFO:teuthology.orchestra.run.smithi187.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-04-24T23:02:40.833 INFO:teuthology.orchestra.run.smithi187.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-24T23:02:40.859 INFO:teuthology.orchestra.run.smithi187.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-24T23:02:40.884 INFO:teuthology.orchestra.run.smithi187.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-24T23:02:40.909 INFO:teuthology.orchestra.run.smithi187.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-04-24T23:02:40.918 INFO:teuthology.orchestra.run.smithi150.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 935 kB/s | 172 kB 00:00 2024-04-24T23:02:40.943 INFO:teuthology.orchestra.run.smithi187.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-24T23:02:40.993 INFO:teuthology.orchestra.run.smithi187.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-24T23:02:41.027 INFO:teuthology.orchestra.run.smithi150.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-04-24T23:02:41.127 INFO:teuthology.orchestra.run.smithi150.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-24T23:02:41.193 INFO:teuthology.orchestra.run.smithi187.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 1.3 MB 00:00 2024-04-24T23:02:41.227 INFO:teuthology.orchestra.run.smithi150.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 982 kB/s | 303 kB 00:00 2024-04-24T23:02:41.286 INFO:teuthology.orchestra.run.smithi150.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-04-24T23:02:41.302 INFO:teuthology.orchestra.run.smithi187.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-24T23:02:41.352 INFO:teuthology.orchestra.run.smithi187.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 7.7 MB/s | 3.4 MB 00:00 2024-04-24T23:02:41.678 INFO:teuthology.orchestra.run.smithi150.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 369 kB/s | 144 kB 00:00 2024-04-24T23:02:41.694 INFO:teuthology.orchestra.run.smithi187.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.2 MB/s | 3.1 MB 00:00 2024-04-24T23:02:41.700 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:41.701 INFO:teuthology.orchestra.run.smithi187.stdout:Total 13 MB/s | 220 MB 00:17 2024-04-24T23:02:41.778 INFO:teuthology.orchestra.run.smithi150.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-24T23:02:41.870 INFO:teuthology.orchestra.run.smithi150.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-04-24T23:02:42.012 INFO:teuthology.orchestra.run.smithi150.stdout:(125/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-04-24T23:02:42.104 INFO:teuthology.orchestra.run.smithi150.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-24T23:02:42.129 INFO:teuthology.orchestra.run.smithi150.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 89 kB/s | 23 kB 00:00 2024-04-24T23:02:42.188 INFO:teuthology.orchestra.run.smithi150.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-24T23:02:42.298 INFO:teuthology.orchestra.run.smithi150.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-04-24T23:02:42.413 INFO:teuthology.orchestra.run.smithi150.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.7 MB/s | 2.0 MB 00:01 2024-04-24T23:02:42.439 INFO:teuthology.orchestra.run.smithi150.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 555 kB/s | 79 kB 00:00 2024-04-24T23:02:42.507 INFO:teuthology.orchestra.run.smithi150.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-04-24T23:02:42.572 INFO:teuthology.orchestra.run.smithi150.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-24T23:02:42.614 INFO:teuthology.orchestra.run.smithi150.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-24T23:02:42.671 INFO:teuthology.orchestra.run.smithi150.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 919 kB/s | 90 kB 00:00 2024-04-24T23:02:42.723 INFO:teuthology.orchestra.run.smithi150.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-24T23:02:42.799 INFO:teuthology.orchestra.run.smithi150.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 241 kB/s | 31 kB 00:00 2024-04-24T23:02:42.823 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-04-24T23:02:42.849 INFO:teuthology.orchestra.run.smithi150.stdout:(138/152): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-04-24T23:02:42.874 INFO:teuthology.orchestra.run.smithi150.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-24T23:02:42.944 INFO:teuthology.orchestra.run.smithi150.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 406 kB/s | 59 kB 00:00 2024-04-24T23:02:42.968 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-04-24T23:02:42.968 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-04-24T23:02:43.033 INFO:teuthology.orchestra.run.smithi150.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 321 kB/s | 51 kB 00:00 2024-04-24T23:02:43.108 INFO:teuthology.orchestra.run.smithi150.stdout:(142/152): python3-tempora-5.0.0-2.el9.noarch.r 138 kB/s | 36 kB 00:00 2024-04-24T23:02:43.242 INFO:teuthology.orchestra.run.smithi150.stdout:(143/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-24T23:02:43.255 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-24T23:02:43.309 INFO:teuthology.orchestra.run.smithi150.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 325 kB/s | 90 kB 00:00 2024-04-24T23:02:43.342 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:02:43.359 INFO:teuthology.orchestra.run.smithi150.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-24T23:02:43.380 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:02:43.434 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:02:43.459 INFO:teuthology.orchestra.run.smithi150.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-24T23:02:43.468 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:02:43.484 INFO:teuthology.orchestra.run.smithi150.stdout:(147/152): python3-webob-1.8.7-6.el9.noarch.rpm 425 kB/s | 230 kB 00:00 2024-04-24T23:02:43.493 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:02:43.536 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:02:43.735 INFO:teuthology.orchestra.run.smithi150.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-04-24T23:02:43.843 INFO:teuthology.orchestra.run.smithi150.stdout:(149/152): librados2-19.0.0-2445.gd2c45d97.el9. 8.8 MB/s | 3.4 MB 00:00 2024-04-24T23:02:43.874 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:02:43.986 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:02:44.061 INFO:teuthology.orchestra.run.smithi150.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 5.4 MB/s | 3.1 MB 00:00 2024-04-24T23:02:44.107 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:02:44.137 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:02:44.228 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:02:44.266 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:02:44.414 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:02:44.464 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:02:44.491 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:02:44.594 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:02:44.617 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:02:44.677 INFO:teuthology.orchestra.run.smithi150.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 1.4 MB/s | 1.3 MB 00:00 2024-04-24T23:02:44.680 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:02:44.702 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:02:44.727 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:02:44.736 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-04-24T23:02:44.737 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-04-24T23:02:44.811 INFO:teuthology.orchestra.run.smithi150.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 2.4 MB 00:00 2024-04-24T23:02:44.813 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:44.814 INFO:teuthology.orchestra.run.smithi150.stdout:Total 10 MB/s | 220 MB 00:21 2024-04-24T23:02:44.885 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:02:44.918 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:02:45.016 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:02:45.113 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:02:45.150 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:02:45.185 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:02:45.250 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:02:45.284 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:02:45.329 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:02:45.361 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:02:45.388 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:02:45.459 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:02:45.520 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:02:45.550 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:02:45.594 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:02:45.689 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:02:45.729 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:02:45.765 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:02:45.793 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:02:45.864 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:02:45.878 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-04-24T23:02:45.908 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:02:45.983 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:02:46.020 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-04-24T23:02:46.021 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-04-24T23:02:46.119 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:02:46.169 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:02:46.708 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:02:46.790 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:02:46.818 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:02:46.860 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:02:46.898 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:02:46.940 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:02:47.105 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:02:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:02:47.206 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:02:47.279 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:02:47.310 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:02:47.347 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:02:47.393 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:02:47.429 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:02:47.472 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:02:47.515 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:02:47.653 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:02:47.704 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:02:47.734 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:02:47.775 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:02:47.778 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-04-24T23:02:47.778 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-04-24T23:02:47.870 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:02:47.900 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:02:47.932 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:02:47.957 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:02:47.982 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:02:48.057 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:02:48.185 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:02:48.223 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:02:48.251 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:02:48.285 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:02:48.324 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:02:48.355 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:02:48.385 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:02:48.420 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:02:48.445 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:02:48.486 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:02:48.522 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:02:48.610 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:02:48.652 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:02:48.764 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:02:48.974 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:02:49.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:02:49.143 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:02:49.874 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:02:49.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:02:50.015 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:02:50.121 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:02:50.173 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:02:50.263 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:02:50.300 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:02:50.476 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:02:50.506 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:02:50.584 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:02:50.638 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-04-24T23:02:50.723 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:02:50.828 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:02:51.000 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:02:51.041 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:02:51.066 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:02:51.110 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:02:51.241 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:02:51.364 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:02:51.443 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:02:51.542 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:02:51.673 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:02:51.701 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:02:51.810 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:02:51.839 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:02:51.991 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:02:52.044 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:02:52.072 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:02:52.167 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:02:52.190 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:02:52.236 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:02:52.257 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:02:52.279 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:02:52.483 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:02:52.517 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:02:52.604 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:02:52.625 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:02:52.685 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:02:52.713 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:02:52.719 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:02:52.747 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:02:52.760 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:02:52.800 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:02:52.864 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:02:52.898 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:02:52.942 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:02:52.974 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:02:53.003 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:02:53.058 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:02:53.071 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:02:53.102 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:02:53.125 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:02:53.163 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:02:53.184 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:02:53.199 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:02:53.214 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:02:53.251 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:02:53.274 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:02:53.318 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:02:53.353 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:02:53.381 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:02:53.451 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:02:53.496 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:02:53.570 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:02:53.704 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:02:53.725 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:02:53.731 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-04-24T23:02:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:02:53.756 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:02:53.814 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-24T23:02:53.826 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:02:53.852 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-24T23:02:53.856 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:02:53.999 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-24T23:02:54.158 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-24T23:02:54.183 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-24T23:02:54.227 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-24T23:02:54.293 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:02:54.376 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:02:54.406 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:02:54.449 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:02:54.487 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:02:54.528 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:02:54.557 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-24T23:02:54.650 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:02:54.672 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:02:54.723 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:02:54.772 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:02:54.778 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:02:54.801 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:02:54.850 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:02:54.882 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:02:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:02:54.918 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:02:54.930 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:02:54.973 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:02:55.010 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:02:55.044 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:02:55.081 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-24T23:02:55.086 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:02:55.136 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-24T23:02:55.163 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:02:55.224 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:02:55.267 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:02:55.267 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:02:55.289 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:02:55.297 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:02:55.339 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:02:55.352 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-24T23:02:55.374 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:02:55.397 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-24T23:02:55.433 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:02:55.472 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:02:55.512 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:02:55.546 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:02:55.570 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:02:55.590 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:02:55.624 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-24T23:02:55.646 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:02:55.732 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-24T23:02:55.781 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:02:55.819 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:02:55.826 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-24T23:02:55.847 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:02:55.864 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-24T23:02:55.882 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:02:55.899 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-24T23:02:55.921 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:02:55.961 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:02:55.964 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-24T23:02:55.998 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-24T23:02:55.999 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:02:56.025 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:02:56.041 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-24T23:02:56.059 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:02:56.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-24T23:02:56.100 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:02:56.102 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-24T23:02:56.120 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:02:56.134 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:02:56.161 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:02:56.173 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-24T23:02:56.224 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:02:56.225 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-24T23:02:56.255 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-24T23:02:56.273 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:02:56.304 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:02:56.340 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-24T23:02:56.427 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:02:56.542 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:02:56.589 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:02:56.598 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-24T23:02:56.625 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-24T23:02:56.660 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-24T23:02:56.689 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-24T23:02:56.705 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:02:56.748 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:02:56.758 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-24T23:02:56.804 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-24T23:02:56.839 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:02:56.839 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:02:56.839 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:02:56.839 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:02:56.866 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:02:56.880 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-24T23:02:57.011 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-24T23:02:57.065 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-24T23:02:57.313 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:02:57.353 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:02:57.387 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:02:57.486 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:02:57.544 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:02:57.597 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-24T23:02:57.632 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:02:57.672 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:02:57.695 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-24T23:02:57.723 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-24T23:02:57.766 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-24T23:02:57.795 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-24T23:02:57.837 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-24T23:02:57.867 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:02:57.904 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:02:57.989 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:02:58.001 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-24T23:02:58.065 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-24T23:02:58.120 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-24T23:02:58.194 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-24T23:02:58.232 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-24T23:02:58.280 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-24T23:02:58.332 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-24T23:02:58.368 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-24T23:02:58.410 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-24T23:02:58.452 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-24T23:02:58.590 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-24T23:02:58.633 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-24T23:02:58.635 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:02:58.663 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-24T23:02:58.705 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-24T23:02:58.743 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:02:58.800 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-24T23:02:58.830 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-24T23:02:58.870 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-24T23:02:58.904 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-24T23:02:58.928 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-24T23:02:59.004 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-24T23:02:59.131 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-24T23:02:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-24T23:02:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-24T23:02:59.223 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-24T23:02:59.262 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-24T23:02:59.293 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-24T23:02:59.323 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-24T23:02:59.362 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-24T23:02:59.391 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-24T23:02:59.433 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-24T23:02:59.467 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-24T23:02:59.556 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-24T23:02:59.597 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-24T23:02:59.629 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-24T23:02:59.676 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-24T23:02:59.776 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-24T23:02:59.831 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-24T23:02:59.969 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:03:00.073 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:03:00.101 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:03:00.134 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:03:00.446 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:03:00.489 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:03:00.561 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-24T23:03:00.579 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:03:00.602 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-24T23:03:00.610 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:03:00.636 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-24T23:03:00.646 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:03:00.736 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-24T23:03:00.793 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-24T23:03:00.881 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-24T23:03:00.913 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-24T23:03:01.093 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-24T23:03:01.118 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:03:01.135 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-24T23:03:01.149 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:03:01.221 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:03:01.247 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-24T23:03:01.251 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:03:01.896 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-24T23:03:02.016 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-24T23:03:03.265 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-24T23:03:03.356 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-24T23:03:03.384 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-24T23:03:03.417 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-24T23:03:03.533 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:03:03.572 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:03:03.730 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-24T23:03:03.796 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:03:03.887 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-24T23:03:03.926 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-24T23:03:03.962 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-24T23:03:04.115 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:03:04.159 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:03:04.248 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:03:04.248 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:03:04.248 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:03:04.248 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:04.277 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:03:04.436 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-24T23:03:04.466 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:03:04.521 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-24T23:03:04.551 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-24T23:03:06.854 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:03:06.897 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:03:07.435 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-24T23:03:07.476 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:03:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-24T23:03:07.560 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:03:07.560 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:03:07.560 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:07.594 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:03:08.223 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:03:08.223 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-04-24T23:03:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-04-24T23:03:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-04-24T23:03:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-04-24T23:03:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-04-24T23:03:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-04-24T23:03:08.224 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-04-24T23:03:08.224 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:08.404 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:03:08.424 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:03:08.424 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:08.424 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:03:08.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:09.882 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:03:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:03:09.908 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:09.908 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:03:09.908 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:03:09.908 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:03:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:10.133 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:03:10.155 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:03:10.156 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:10.156 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:03:10.156 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:03:10.156 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:03:10.156 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:10.600 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:03:10.622 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:03:10.623 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:10.623 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:03:10.623 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:03:10.623 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:03:10.623 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:10.730 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:03:10.788 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:03:10.842 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:03:11.028 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:03:11.084 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:03:11.162 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:03:11.251 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:03:11.391 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:03:11.426 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:03:11.458 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:03:11.486 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:03:11.918 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:03:11.951 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:03:11.983 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:03:12.044 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:03:12.082 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:03:13.117 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:03:13.151 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:03:14.113 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:03:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:03:14.237 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:03:14.398 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:03:14.437 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:03:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:03:14.462 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:14.462 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:03:14.462 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:03:14.462 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:03:14.462 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:14.504 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:03:14.516 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:03:14.612 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:03:14.660 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:03:15.554 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:03:15.590 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:03:15.613 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:03:15.613 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:15.613 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:03:15.613 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:03:15.613 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:03:15.613 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:15.657 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:03:15.680 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:03:15.680 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:15.680 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:03:15.680 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:15.733 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:03:15.747 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:03:15.747 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-04-24T23:03:15.748 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-04-24T23:03:15.748 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-04-24T23:03:15.748 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-04-24T23:03:15.748 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-04-24T23:03:15.748 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-04-24T23:03:15.748 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-04-24T23:03:15.748 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:15.755 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:03:15.755 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:15.755 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:03:15.755 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:03:15.755 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:03:15.755 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:15.914 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:03:15.930 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:03:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:15.930 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:03:15.930 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:16.059 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:03:16.081 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:03:16.081 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:16.081 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:03:16.081 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:03:16.081 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:03:16.082 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:17.417 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:03:17.442 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:03:17.443 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:17.443 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:03:17.443 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:03:17.443 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:03:17.443 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:17.667 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:03:17.690 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:03:17.690 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:17.690 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:03:17.690 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:03:17.690 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:03:17.690 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:18.127 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:03:18.149 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:03:18.149 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:18.149 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:03:18.149 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:03:18.149 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:03:18.150 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:18.248 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:03:18.305 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:03:18.363 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:03:18.544 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:03:18.594 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:03:18.672 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:03:18.760 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:03:18.901 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:03:18.944 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:03:18.978 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:03:19.003 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:03:19.401 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-24T23:03:19.401 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-04-24T23:03:19.401 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-04-24T23:03:19.401 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-04-24T23:03:19.401 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-04-24T23:03:19.401 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-04-24T23:03:19.401 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-04-24T23:03:19.402 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-04-24T23:03:19.402 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:19.442 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:03:19.468 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:03:19.500 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:03:19.561 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:03:19.564 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:03:19.580 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-24T23:03:19.580 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:19.580 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-24T23:03:19.580 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:19.600 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:03:20.579 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:03:20.621 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:03:20.629 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:03:20.661 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:03:20.668 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:03:20.693 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:03:20.721 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:03:20.904 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:03:20.954 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:03:20.980 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:03:21.076 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:03:21.101 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-24T23:03:21.102 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:21.102 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:03:21.102 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:03:21.102 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:03:21.102 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:21.318 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:03:21.340 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-24T23:03:21.341 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:21.341 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:03:21.341 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:03:21.341 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:03:21.341 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:21.568 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:21.603 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:21.624 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:21.638 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:03:21.668 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:03:21.769 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:03:21.804 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:03:21.825 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-24T23:03:21.825 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:21.825 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:03:21.825 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:03:21.826 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:03:21.826 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:21.932 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:03:21.932 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-24T23:03:21.962 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:03:21.980 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-24T23:03:21.985 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:03:21.985 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:21.985 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:03:21.985 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:03:21.985 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:03:21.985 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:22.029 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:03:22.036 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-24T23:03:22.040 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:03:22.137 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:03:22.186 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:03:22.222 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-24T23:03:22.270 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-24T23:03:22.356 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-24T23:03:22.444 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:03:22.462 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:03:22.550 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:03:22.584 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:03:22.591 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:03:22.628 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-24T23:03:22.663 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-24T23:03:22.672 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:03:22.687 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-24T23:03:22.704 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:03:22.704 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:03:22.788 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:03:22.813 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:03:22.813 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:03:22.906 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:03:22.980 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:23.083 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:03:23.111 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:03:23.115 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:03:23.137 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:03:23.138 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:23.138 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:03:23.138 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:03:23.138 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:03:23.138 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:23.144 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-24T23:03:23.176 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-24T23:03:23.182 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:03:23.204 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:03:23.204 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:23.204 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:03:23.204 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:23.238 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-24T23:03:23.249 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:03:23.267 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:03:23.272 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:03:23.272 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:23.272 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:03:23.272 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:03:23.272 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:03:23.272 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:23.575 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:03:23.597 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:03:23.598 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:23.598 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:03:23.598 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:03:23.598 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:03:23.598 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:24.307 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-24T23:03:24.344 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:03:25.311 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-24T23:03:25.344 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:03:25.446 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-24T23:03:25.609 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-24T23:03:25.638 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:03:25.661 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-24T23:03:25.661 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:25.661 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:03:25.661 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:03:25.661 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:03:25.662 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:25.705 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:03:25.717 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-24T23:03:25.815 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-24T23:03:25.862 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-24T23:03:26.760 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-24T23:03:26.800 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:03:26.823 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-24T23:03:26.823 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:26.823 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:03:26.823 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:03:26.823 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:03:26.823 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:26.867 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:03:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-24T23:03:26.890 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:26.890 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:03:26.890 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:26.934 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:03:26.957 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-24T23:03:26.957 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:26.958 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-24T23:03:26.958 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:03:26.958 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-24T23:03:26.958 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:27.260 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:03:27.283 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-24T23:03:27.283 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:03:27.283 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:03:27.283 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:03:27.283 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:03:27.283 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:28.104 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:03:28.145 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:03:28.186 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:03:28.219 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:03:28.246 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:03:28.418 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:03:28.458 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:03:28.489 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:03:28.797 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:28.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:03:28.797 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:03:28.798 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:03:28.838 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:03:28.838 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:03:28.838 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:03:28.838 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:03:28.838 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:03:28.838 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:03:28.838 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:03:28.839 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:03:28.840 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:03:28.841 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:03:28.842 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:03:28.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:03:28.844 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:03:28.845 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:03:28.846 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:03:28.847 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:03:28.848 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:03:28.849 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:03:29.050 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:29.094 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:29.115 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:29.950 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:03:30.041 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:03:30.081 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:03:30.162 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:03:30.195 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:03:30.195 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:03:30.279 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:03:30.312 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:03:30.312 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:03:30.405 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:03:30.476 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:30.843 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:30.843 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:30.843 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-04-24T23:03:30.843 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.843 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.843 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.844 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.845 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:03:30.846 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:03:30.847 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:03:30.848 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:03:30.849 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:03:30.850 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:03:30.851 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:03:30.852 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.853 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:03:30.854 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:03:30.855 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:03:30.856 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:03:30.856 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:03:30.856 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:03:30.856 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:03:30.856 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-24T23:03:31.228 DEBUG:teuthology.parallel:result is None 2024-04-24T23:03:31.789 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-24T23:03:31.830 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-24T23:03:31.871 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-24T23:03:31.903 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-24T23:03:31.931 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-24T23:03:32.105 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-24T23:03:32.152 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-24T23:03:32.182 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-24T23:03:32.745 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:32.788 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:32.807 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-24T23:03:33.656 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-24T23:03:33.742 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-24T23:03:33.784 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-24T23:03:33.872 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-24T23:03:33.913 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-24T23:03:33.913 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:03:33.990 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-24T23:03:34.016 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:03:34.018 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:03:34.131 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-24T23:03:34.195 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:36.435 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:36.435 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:03:36.436 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:03:36.437 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:03:36.438 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:03:36.439 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:03:36.440 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:03:36.441 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:03:36.442 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:03:36.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:03:36.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:03:36.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:03:36.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:03:36.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:03:36.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:03:36.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:03:36.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:03:36.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:03:36.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:03:38.492 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:38.492 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:38.493 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-04-24T23:03:38.493 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.493 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.493 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:38.493 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:38.493 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-04-24T23:03:38.493 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:03:38.494 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:03:38.494 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:03:38.494 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:03:38.494 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.494 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.494 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.494 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.494 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.495 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.495 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.495 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.495 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.495 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.495 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.495 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.496 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.496 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.496 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.496 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.496 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.496 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.497 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.497 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.497 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.497 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.497 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:03:38.498 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:03:38.499 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:03:38.499 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:03:38.499 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.499 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.499 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.499 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:03:38.499 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:03:38.499 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:03:38.500 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:03:38.501 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:03:38.501 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:03:38.501 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:03:38.501 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:03:38.501 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:03:38.501 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:03:38.501 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:03:38.502 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:38.502 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:03:38.502 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:03:38.502 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:03:38.502 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:03:38.502 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:03:38.503 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:03:38.503 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:03:38.503 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:03:38.503 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:03:38.503 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:03:38.504 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:03:38.505 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:03:38.505 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:03:38.505 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:03:38.505 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:03:38.505 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:03:38.505 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:03:38.505 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:03:38.505 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:03:38.506 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:03:38.506 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:03:38.506 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:03:38.506 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:03:38.506 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:03:38.506 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:03:38.506 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:03:38.507 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:03:38.507 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:03:38.507 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:03:38.507 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:03:38.507 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:03:38.507 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:03:38.507 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:03:38.508 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:03:38.508 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:03:38.508 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:03:38.508 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:03:38.508 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:03:38.508 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:03:38.509 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:03:38.509 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:03:38.509 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:03:38.509 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:03:38.509 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:03:38.510 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:03:38.510 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:03:38.510 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.510 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.510 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:03:38.510 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:03:38.510 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:03:38.510 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:03:38.511 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:03:38.512 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:03:38.512 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:03:38.512 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:03:38.512 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:03:38.512 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:03:38.512 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.512 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.512 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:38.513 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:03:38.513 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:03:38.513 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:03:38.513 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:03:38.513 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:03:38.513 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:03:38.514 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:03:38.514 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:03:38.514 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-04-24T23:03:38.885 DEBUG:teuthology.parallel:result is None 2024-04-24T23:03:39.992 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-24T23:03:39.993 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-24T23:03:39.994 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-24T23:03:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-24T23:03:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-24T23:03:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-24T23:03:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-24T23:03:39.995 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-24T23:03:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-24T23:03:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-24T23:03:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-24T23:03:40.007 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-24T23:03:40.008 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-24T23:03:40.009 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-24T23:03:40.010 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-24T23:03:40.011 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-24T23:03:40.012 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-24T23:03:40.013 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-24T23:03:40.014 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-24T23:03:40.015 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-24T23:03:40.016 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-24T23:03:40.017 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-24T23:03:40.017 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-04-24T23:03:42.106 INFO:teuthology.orchestra.run.smithi150.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.2-8.el9.noarch 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.7.4-5.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.107 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.108 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.4-9.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.109 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:03:42.110 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: lua-5.4.4-4.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-13.14-1.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-24T23:03:42.111 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:03:42.112 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:03:42.113 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:03:42.114 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-24T23:03:42.116 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: re2-1:20211101-3.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-24T23:03:42.117 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-24T23:03:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:03:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-24T23:03:42.118 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-04-24T23:03:42.118 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-04-24T23:03:42.496 DEBUG:teuthology.parallel:result is None 2024-04-24T23:03:42.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:03:42.637 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:03:42.663 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:03:42.663 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:03:42.664 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:03:42.665 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:03:42.882 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:03:42.903 INFO:teuthology.orchestra.run.smithi150.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:03:42.903 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:03:42.903 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:03:42.906 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:03:43.122 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:03:43.148 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-24T23:03:43.148 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-24T23:03:43.148 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-24T23:03:43.150 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:03:43.150 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:03:43.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:03:43.181 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:03:43.181 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:03:43.208 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:03:43.208 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:03:43.238 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:03:43.239 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:03:43.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:03:43.267 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:03:43.336 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:03:43.336 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:03:43.360 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:03:43.423 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:03:43.423 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:03:43.451 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:03:43.519 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:03:43.519 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:03:43.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:03:43.547 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:03:43.614 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:03:43.614 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:03:43.637 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:03:43.701 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:03:43.701 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:03:43.728 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:03:43.796 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:03:43.797 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:03:43.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:03:43.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:03:43.892 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:03:43.893 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:03:43.918 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:03:43.981 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:03:43.981 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:03:44.009 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:03:44.078 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T23:03:44.199 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client dirsize rbytes': False, 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-24T23:03:44.200 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:03:44.200 INFO:tasks.cephadm:Cluster fsid is e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:44.200 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T23:03:44.201 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.112', 'mon.b': '172.21.15.150', 'mon.c': '172.21.15.187'} 2024-04-24T23:03:44.201 INFO:tasks.cephadm:First mon is mon.a on smithi112 2024-04-24T23:03:44.201 INFO:tasks.cephadm:First mgr is x 2024-04-24T23:03:44.201 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T23:03:44.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-04-24T23:03:44.230 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-04-24T23:03:44.255 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname $(hostname -s) 2024-04-24T23:03:44.284 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-24T23:03:44.284 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:03:44.501 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-24T23:03:44.721 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:03:44.722 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-24T23:03:44.723 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:45.303 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:03 /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:45.303 DEBUG:teuthology.orchestra.run.smithi150:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:45.885 INFO:teuthology.orchestra.run.smithi150.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:03 /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:45.886 DEBUG:teuthology.orchestra.run.smithi187:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:46.267 INFO:teuthology.orchestra.run.smithi187.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 24 23:03 /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:46.267 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:46.286 DEBUG:teuthology.orchestra.run.smithi150:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:46.304 DEBUG:teuthology.orchestra.run.smithi187:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:03:46.339 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-24T23:03:46.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:03:46.342 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:03:46.346 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-24T23:03:47.918 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:03:48.013 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:03:48.014 INFO:teuthology.orchestra.run.smithi187.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:04:15.732 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:04:15.732 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:04:15.732 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:04:15.733 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-04-24T23:04:15.733 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:04:15.733 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:04:15.733 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-24T23:04:19.311 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:04:19.312 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:04:19.312 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-04-24T23:04:19.312 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:04:19.312 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-04-24T23:04:19.312 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-24T23:04:19.621 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-24T23:04:19.622 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:04:19.622 INFO:teuthology.orchestra.run.smithi187.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-24T23:04:19.622 INFO:teuthology.orchestra.run.smithi187.stdout: "repo_digests": [ 2024-04-24T23:04:19.622 INFO:teuthology.orchestra.run.smithi187.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-24T23:04:19.622 INFO:teuthology.orchestra.run.smithi187.stdout: ] 2024-04-24T23:04:19.622 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-24T23:04:19.646 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-04-24T23:04:19.678 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-04-24T23:04:19.706 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph 2024-04-24T23:04:19.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-04-24T23:04:19.760 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-04-24T23:04:19.785 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /etc/ceph 2024-04-24T23:04:19.808 INFO:tasks.cephadm:Writing seed config... 2024-04-24T23:04:19.809 INFO:tasks.cephadm: override: [client] client dirsize rbytes = False 2024-04-24T23:04:19.809 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T23:04:19.809 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T23:04:19.809 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T23:04:19.809 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T23:04:19.809 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T23:04:19.810 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T23:04:19.811 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T23:04:19.812 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T23:04:19.813 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T23:04:19.814 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T23:04:19.814 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T23:04:19.814 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T23:04:19.814 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-24T23:04:19.814 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-24T23:04:19.815 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:04:19.815 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T23:04:19.832 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e62ba378-028e-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client dirsize rbytes = False client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T23:04:19.833 DEBUG:teuthology.orchestra.run.smithi112:mon.a> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a.service 2024-04-24T23:04:19.875 DEBUG:teuthology.orchestra.run.smithi112:mgr.x> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.x.service 2024-04-24T23:04:19.918 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T23:04:19.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid e62ba378-028e-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.112 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:20.110 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:04:20.111 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', 'e62ba378-028e-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.112', '--skip-admin-label'] 2024-04-24T23:04:20.111 INFO:teuthology.orchestra.run.smithi112.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T23:04:20.111 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying podman|docker is present... 2024-04-24T23:04:20.137 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:04:20.137 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying lvm2 is present... 2024-04-24T23:04:20.137 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying time synchronization is in place... 2024-04-24T23:04:20.144 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:04:20.144 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:04:20.150 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:04:20.150 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-24T23:04:20.157 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-04-24T23:04:20.163 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-04-24T23:04:20.163 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:04:20.163 INFO:teuthology.orchestra.run.smithi112.stdout:Repeating the final host check... 2024-04-24T23:04:20.193 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:04:20.194 INFO:teuthology.orchestra.run.smithi112.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T23:04:20.194 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl is present 2024-04-24T23:04:20.194 INFO:teuthology.orchestra.run.smithi112.stdout:lvcreate is present 2024-04-24T23:04:20.200 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:04:20.200 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:04:20.206 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:04:20.206 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-24T23:04:20.211 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-04-24T23:04:20.218 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-04-24T23:04:20.218 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:04:20.218 INFO:teuthology.orchestra.run.smithi112.stdout:Host looks OK 2024-04-24T23:04:20.218 INFO:teuthology.orchestra.run.smithi112.stdout:Cluster fsid: e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:20.218 INFO:teuthology.orchestra.run.smithi112.stdout:Acquiring lock 140547063514016 on /run/cephadm/e62ba378-028e-11ef-bc93-c7b262605968.lock 2024-04-24T23:04:20.218 INFO:teuthology.orchestra.run.smithi112.stdout:Lock 140547063514016 acquired on /run/cephadm/e62ba378-028e-11ef-bc93-c7b262605968.lock 2024-04-24T23:04:20.218 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 3300 ... 2024-04-24T23:04:20.219 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 6789 ... 2024-04-24T23:04:20.219 INFO:teuthology.orchestra.run.smithi112.stdout:Base mon IP(s) is [172.21.15.112:3300, 172.21.15.112:6789], mon addrv is [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-04-24T23:04:20.222 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.112 metric 100 2024-04-24T23:04:20.222 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.112 metric 100 2024-04-24T23:04:20.224 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T23:04:20.225 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-24T23:04:20.225 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-24T23:04:20.227 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T23:04:20.227 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T23:04:20.227 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:04:20.227 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-24T23:04:20.227 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 fe80::5787:d070:61ba:675f/64 scope link noprefixroute 2024-04-24T23:04:20.228 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:04:20.228 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-04-24T23:04:20.228 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-04-24T23:04:20.229 INFO:teuthology.orchestra.run.smithi112.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T23:04:20.229 INFO:teuthology.orchestra.run.smithi112.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T23:04:20.230 INFO:teuthology.orchestra.run.smithi112.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:04:20.676 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:04:20.676 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-24T23:04:20.676 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T23:04:20.677 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-24T23:04:20.677 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T23:04:20.677 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-24T23:04:20.677 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T23:04:21.648 INFO:teuthology.orchestra.run.smithi112.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:04:21.648 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-24T23:04:21.648 INFO:teuthology.orchestra.run.smithi112.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T23:04:22.676 INFO:teuthology.orchestra.run.smithi112.stdout:stat: stdout 167 167 2024-04-24T23:04:22.676 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial keys... 2024-04-24T23:04:23.594 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQD3jylmWAwUABAAsiMmgJTjHHOJ0veTvNePcg== 2024-04-24T23:04:24.526 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQD3jylmrs5UOBAAMQl6SNjVz5XY956fakcCuw== 2024-04-24T23:04:25.540 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQD4jylmS27NMhAAL6QFdvNMNimYnm5MfLa9tg== 2024-04-24T23:04:25.541 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial monmap... 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool for a [v2:172.21.15.112:3300,v1:172.21.15.112:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: set fsid to e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:04:26.413 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:04:26.414 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mon... 2024-04-24T23:04:27.428 INFO:teuthology.orchestra.run.smithi112.stdout:create mon.a on 2024-04-24T23:04:27.619 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T23:04:27.761 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T23:04:27.926 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e62ba378-028e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-e62ba378-028e-11ef-bc93-c7b262605968.target. 2024-04-24T23:04:27.926 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e62ba378-028e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-e62ba378-028e-11ef-bc93-c7b262605968.target. 2024-04-24T23:04:28.176 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:04:28.176 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-24T23:04:28.344 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e62ba378-028e-11ef-bc93-c7b262605968.target.wants/ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-e62ba378-028e-11ef-bc93-c7b262605968@.service. 2024-04-24T23:04:28.901 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:04:28.902 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-24T23:04:28.908 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:04:28.908 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-24T23:04:28.908 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-24T23:04:28.908 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:04:28.909 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon to start... 2024-04-24T23:04:28.909 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon... 2024-04-24T23:04:29.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:28 smithi112 systemd[1]: Started Ceph mon.a for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: monmap epoch 1 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: last_changed 2024-04-24T23:04:25.846546+0000 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: min_mon_release 19 (squid) 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: election_strategy: 1 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: fsmap 2024-04-24T23:04:29.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:04:29.964 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: mgrmap e1: no daemons active 2024-04-24T23:04:29.964 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:29 smithi112 ceph-mon[28814]: from='client.? 172.21.15.112:0/2766719128' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T23:04:29.965 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T23:04:29.971 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout id: e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:29.971 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T23:04:29.971 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout services: 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.567006s) 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout data: 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.339+0000 7fd2de6e8700 1 Processor -- start 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.339+0000 7fd2de6e8700 1 -- start start 2024-04-24T23:04:29.972 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.339+0000 7fd2de6e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d810ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.339+0000 7fd2de6e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d807bdf0 con 0x7fd2d8108740 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.340+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d810ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.340+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d810ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42648/0 (socket says 172.21.15.112:42648) 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.340+0000 7fd2d7fff700 1 -- 172.21.15.112:0/1817891598 learned_addr learned my addr 172.21.15.112:0/1817891598 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.340+0000 7fd2d7fff700 1 -- 172.21.15.112:0/1817891598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d810b070 con 0x7fd2d8108740 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.340+0000 7fd2d7fff700 1 --2- 172.21.15.112:0/1817891598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d810ab30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80098a0 tx=0x7fd2c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27eab6cc2563e736 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2d6ffd700 1 -- 172.21.15.112:0/1817891598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c8004160 con 0x7fd2d8108740 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2d6ffd700 1 -- 172.21.15.112:0/1817891598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c8004300 con 0x7fd2d8108740 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2d6ffd700 1 -- 172.21.15.112:0/1817891598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c80036a0 con 0x7fd2d8108740 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2de6e8700 1 -- 172.21.15.112:0/1817891598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 msgr2=0x7fd2d810ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2de6e8700 1 --2- 172.21.15.112:0/1817891598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d810ab30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80098a0 tx=0x7fd2c8009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2de6e8700 1 -- 172.21.15.112:0/1817891598 shutdown_connections 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2de6e8700 1 --2- 172.21.15.112:0/1817891598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d810ab30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:29.973 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2de6e8700 1 -- 172.21.15.112:0/1817891598 >> 172.21.15.112:0/1817891598 conn(0x7fd2d8102170 msgr2=0x7fd2d8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2de6e8700 1 -- 172.21.15.112:0/1817891598 shutdown_connections 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.341+0000 7fd2de6e8700 1 -- 172.21.15.112:0/1817891598 wait complete. 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.342+0000 7fd2de6e8700 1 Processor -- start 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.342+0000 7fd2de6e8700 1 -- start start 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.342+0000 7fd2de6e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.342+0000 7fd2de6e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d810b4e0 con 0x7fd2d8108740 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.342+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.342+0000 7fd2d7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42654/0 (socket says 172.21.15.112:42654) 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.342+0000 7fd2d7fff700 1 -- 172.21.15.112:0/2766719128 learned_addr learned my addr 172.21.15.112:0/2766719128 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2d7fff700 1 -- 172.21.15.112:0/2766719128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d819b920 con 0x7fd2d8108740 2024-04-24T23:04:29.974 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2d7fff700 1 --2- 172.21.15.112:0/2766719128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d819b3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8000c00 tx=0x7fd2c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2d57fa700 1 -- 172.21.15.112:0/2766719128 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c8003db0 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2d57fa700 1 -- 172.21.15.112:0/2766719128 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c8011630 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2d57fa700 1 -- 172.21.15.112:0/2766719128 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c8011950 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d819bbb0 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d819e8a0 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2d57fa700 1 -- 172.21.15.112:0/2766719128 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd2c801a460 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.343+0000 7fd2d57fa700 1 -- 172.21.15.112:0/2766719128 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd2c801bb40 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.344+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d8063890 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.346+0000 7fd2d57fa700 1 -- 172.21.15.112:0/2766719128 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd2c8004260 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.421+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd2d802dd80 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.422+0000 7fd2d57fa700 1 -- 172.21.15.112:0/2766719128 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd2c801bd90 con 0x7fd2d8108740 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.423+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 msgr2=0x7fd2d819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.423+0000 7fd2de6e8700 1 --2- 172.21.15.112:0/2766719128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d819b3e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8000c00 tx=0x7fd2c8004750 comp rx=0 tx=0).stop 2024-04-24T23:04:29.975 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.423+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 shutdown_connections 2024-04-24T23:04:29.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.423+0000 7fd2de6e8700 1 --2- 172.21.15.112:0/2766719128 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2d8108740 0x7fd2d819b3e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:29.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.423+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 >> 172.21.15.112:0/2766719128 conn(0x7fd2d8102170 msgr2=0x7fd2d81044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:29.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.423+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 shutdown_connections 2024-04-24T23:04:29.976 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.423+0000 7fd2de6e8700 1 -- 172.21.15.112:0/2766719128 wait complete. 2024-04-24T23:04:29.976 INFO:teuthology.orchestra.run.smithi112.stdout:mon is available 2024-04-24T23:04:29.976 INFO:teuthology.orchestra.run.smithi112.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T23:04:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:30 smithi112 ceph-mon[28814]: from='client.? 172.21.15.112:0/906694068' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:04:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:30 smithi112 ceph-mon[28814]: from='client.? 172.21.15.112:0/906694068' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T23:04:31.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:31.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:04:31.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:31.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-04-24T23:04:31.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:04:31.061 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:04:31.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:04:31.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:04:31.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:31.062 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:04:31.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:04:31.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.388+0000 7fe958e7b700 1 Processor -- start 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.388+0000 7fe958e7b700 1 -- start start 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.388+0000 7fe958e7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe954107000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.388+0000 7fe958e7b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe9541075d0 con 0x7fe954106be0 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.388+0000 7fe95259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe954107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.388+0000 7fe95259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe954107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42660/0 (socket says 172.21.15.112:42660) 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.388+0000 7fe95259c700 1 -- 172.21.15.112:0/20976131 learned_addr learned my addr 172.21.15.112:0/20976131 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:31.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe95259c700 1 -- 172.21.15.112:0/20976131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe954107de0 con 0x7fe954106be0 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe95259c700 1 --2- 172.21.15.112:0/20976131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe954107000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe9440098a0 tx=0x7fe944009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a9b73969d52c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe951d9b700 1 -- 172.21.15.112:0/20976131 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe944004160 con 0x7fe954106be0 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe951d9b700 1 -- 172.21.15.112:0/20976131 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe944004300 con 0x7fe954106be0 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe951d9b700 1 -- 172.21.15.112:0/20976131 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9440036a0 con 0x7fe954106be0 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe958e7b700 1 -- 172.21.15.112:0/20976131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 msgr2=0x7fe954107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe958e7b700 1 --2- 172.21.15.112:0/20976131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe954107000 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe9440098a0 tx=0x7fe944009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe958e7b700 1 -- 172.21.15.112:0/20976131 shutdown_connections 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe958e7b700 1 --2- 172.21.15.112:0/20976131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe954107000 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.389+0000 7fe958e7b700 1 -- 172.21.15.112:0/20976131 >> 172.21.15.112:0/20976131 conn(0x7fe9541021a0 msgr2=0x7fe9541045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.390+0000 7fe958e7b700 1 -- 172.21.15.112:0/20976131 shutdown_connections 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.390+0000 7fe958e7b700 1 -- 172.21.15.112:0/20976131 wait complete. 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.390+0000 7fe958e7b700 1 Processor -- start 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.390+0000 7fe958e7b700 1 -- start start 2024-04-24T23:04:31.069 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe958e7b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe95419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe958e7b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe954108310 con 0x7fe954106be0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe95259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe95419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe95259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe95419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42662/0 (socket says 172.21.15.112:42662) 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe95259c700 1 -- 172.21.15.112:0/906694068 learned_addr learned my addr 172.21.15.112:0/906694068 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe95259c700 1 -- 172.21.15.112:0/906694068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95419b940 con 0x7fe954106be0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe95259c700 1 --2- 172.21.15.112:0/906694068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe95419b400 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe944000c00 tx=0x7fe944004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe94bfff700 1 -- 172.21.15.112:0/906694068 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe944003db0 con 0x7fe954106be0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe94bfff700 1 -- 172.21.15.112:0/906694068 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe944011630 con 0x7fe954106be0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95419bbd0 con 0x7fe954106be0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe94bfff700 1 -- 172.21.15.112:0/906694068 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe944011950 con 0x7fe954106be0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95419e8c0 con 0x7fe954106be0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe94bfff700 1 -- 172.21.15.112:0/906694068 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe94401a460 con 0x7fe954106be0 2024-04-24T23:04:31.070 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.391+0000 7fe94bfff700 1 -- 172.21.15.112:0/906694068 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe94401b780 con 0x7fe954106be0 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.392+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe954063890 con 0x7fe954106be0 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.395+0000 7fe94bfff700 1 -- 172.21.15.112:0/906694068 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe944031020 con 0x7fe954106be0 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.462+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe95419c0b0 con 0x7fe954106be0 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.465+0000 7fe94bfff700 1 -- 172.21.15.112:0/906694068 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fe95419c0b0 con 0x7fe954106be0 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.466+0000 7fe94bfff700 1 -- 172.21.15.112:0/906694068 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fe94401a8e0 con 0x7fe954106be0 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.466+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 msgr2=0x7fe95419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.466+0000 7fe958e7b700 1 --2- 172.21.15.112:0/906694068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe95419b400 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe944000c00 tx=0x7fe944004750 comp rx=0 tx=0).stop 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.466+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 shutdown_connections 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.466+0000 7fe958e7b700 1 --2- 172.21.15.112:0/906694068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe954106be0 0x7fe95419b400 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.466+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 >> 172.21.15.112:0/906694068 conn(0x7fe9541021a0 msgr2=0x7fe954192000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.466+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 shutdown_connections 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:30.466+0000 7fe958e7b700 1 -- 172.21.15.112:0/906694068 wait complete. 2024-04-24T23:04:31.071 INFO:teuthology.orchestra.run.smithi112.stdout:Generating new minimal ceph.conf... 2024-04-24T23:04:32.123 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:31 smithi112 ceph-mon[28814]: from='client.? 172.21.15.112:0/2592965989' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:04:32.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.460+0000 7fddc9d47700 1 Processor -- start 2024-04-24T23:04:32.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.460+0000 7fddc9d47700 1 -- start start 2024-04-24T23:04:32.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.460+0000 7fddc9d47700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc41069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:32.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.460+0000 7fddc9d47700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fddc407bd90 con 0x7fddc41065d0 2024-04-24T23:04:32.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.461+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc41069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:32.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.461+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc41069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39460/0 (socket says 172.21.15.112:39460) 2024-04-24T23:04:32.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.461+0000 7fddc37fe700 1 -- 172.21.15.112:0/1508895380 learned_addr learned my addr 172.21.15.112:0/1508895380 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.461+0000 7fddc37fe700 1 -- 172.21.15.112:0/1508895380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc4106f30 con 0x7fddc41065d0 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.461+0000 7fddc37fe700 1 --2- 172.21.15.112:0/1508895380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc41069f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fddb40098a0 tx=0x7fddb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e0762b655ac224 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.461+0000 7fddc2ffd700 1 -- 172.21.15.112:0/1508895380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb4004160 con 0x7fddc41065d0 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.461+0000 7fddc2ffd700 1 -- 172.21.15.112:0/1508895380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fddb4004300 con 0x7fddc41065d0 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.461+0000 7fddc2ffd700 1 -- 172.21.15.112:0/1508895380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb4011c00 con 0x7fddc41065d0 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.462+0000 7fddc9d47700 1 -- 172.21.15.112:0/1508895380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 msgr2=0x7fddc41069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.462+0000 7fddc9d47700 1 --2- 172.21.15.112:0/1508895380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc41069f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fddb40098a0 tx=0x7fddb4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.462+0000 7fddc9d47700 1 -- 172.21.15.112:0/1508895380 shutdown_connections 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.462+0000 7fddc9d47700 1 --2- 172.21.15.112:0/1508895380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc41069f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.462+0000 7fddc9d47700 1 -- 172.21.15.112:0/1508895380 >> 172.21.15.112:0/1508895380 conn(0x7fddc4102190 msgr2=0x7fddc41045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.462+0000 7fddc9d47700 1 -- 172.21.15.112:0/1508895380 shutdown_connections 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.462+0000 7fddc9d47700 1 -- 172.21.15.112:0/1508895380 wait complete. 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.462+0000 7fddc9d47700 1 Processor -- start 2024-04-24T23:04:32.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc9d47700 1 -- start start 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc9d47700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc9d47700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fddc4108020 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc419def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39468/0 (socket says 172.21.15.112:39468) 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc37fe700 1 -- 172.21.15.112:0/2592965989 learned_addr learned my addr 172.21.15.112:0/2592965989 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc37fe700 1 -- 172.21.15.112:0/2592965989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc419e430 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc37fe700 1 --2- 172.21.15.112:0/2592965989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc419def0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009fb0 tx=0x7fddb4011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.463+0000 7fddc17fa700 1 -- 172.21.15.112:0/2592965989 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb40038b0 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.464+0000 7fddc17fa700 1 -- 172.21.15.112:0/2592965989 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fddb401a430 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.464+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc419e6c0 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.464+0000 7fddc17fa700 1 -- 172.21.15.112:0/2592965989 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddb4019630 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.464+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc41a13b0 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.464+0000 7fddc17fa700 1 -- 172.21.15.112:0/2592965989 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fddb4019c70 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.464+0000 7fddc17fa700 1 -- 172.21.15.112:0/2592965989 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fddb4024730 con 0x7fddc41065d0 2024-04-24T23:04:32.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.465+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc4063890 con 0x7fddc41065d0 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.467+0000 7fddc17fa700 1 -- 172.21.15.112:0/2592965989 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fddb401aa10 con 0x7fddc41065d0 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.534+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fddc40008d0 con 0x7fddc41065d0 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.535+0000 7fddc17fa700 1 -- 172.21.15.112:0/2592965989 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7fddb4032030 con 0x7fddc41065d0 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.535+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 msgr2=0x7fddc419def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.535+0000 7fddc9d47700 1 --2- 172.21.15.112:0/2592965989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc419def0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009fb0 tx=0x7fddb4011c60 comp rx=0 tx=0).stop 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.536+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 shutdown_connections 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.536+0000 7fddc9d47700 1 --2- 172.21.15.112:0/2592965989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddc41065d0 0x7fddc419def0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.536+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 >> 172.21.15.112:0/2592965989 conn(0x7fddc4102190 msgr2=0x7fddc4194b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.536+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 shutdown_connections 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.536+0000 7fddc9d47700 1 -- 172.21.15.112:0/2592965989 wait complete. 2024-04-24T23:04:32.133 INFO:teuthology.orchestra.run.smithi112.stdout:Restarting the monitor... 2024-04-24T23:04:32.394 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:32 smithi112 systemd[1]: Stopping Ceph mon.a for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:04:32.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:32 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a[28810]: 2024-04-24T23:04:32.391+0000 7fa361dcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:04:32.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:32 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a[28810]: 2024-04-24T23:04:32.391+0000 7fa361dcb700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T23:04:32.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:32 smithi112 podman[29060]: 2024-04-24 23:04:32.638240844 +0000 UTC m=+0.340915825 container died f7d9aed23f74d1560ad3ea15a2292d1cc2f641b54f68f4a2e05b0aef4e06d6f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T23:04:33.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:32 smithi112 podman[29060]: 2024-04-24 23:04:32.975105679 +0000 UTC m=+0.677780660 container cleanup f7d9aed23f74d1560ad3ea15a2292d1cc2f641b54f68f4a2e05b0aef4e06d6f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-24T23:04:33.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:32 smithi112 bash[29060]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:04:33.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 podman[29074]: 2024-04-24 23:04:33.134171681 +0000 UTC m=+0.499460099 container remove f7d9aed23f74d1560ad3ea15a2292d1cc2f641b54f68f4a2e05b0aef4e06d6f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-24T23:04:33.633 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T23:04:33.633 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 systemd[1]: Stopped Ceph mon.a for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:33.633 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 systemd[1]: Starting Ceph mon.a for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:04:33.633 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 podman[29167]: 2024-04-24 23:04:33.551702153 +0000 UTC m=+0.108567252 container create d2f995aeef1b83922afd350e42bb086710d07ca86cb6101e3309ad300d098f3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T23:04:33.633 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 podman[29167]: 2024-04-24 23:04:33.461208054 +0000 UTC m=+0.018073117 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:04:33.809 INFO:teuthology.orchestra.run.smithi112.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T23:04:34.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 podman[29167]: 2024-04-24 23:04:33.72897413 +0000 UTC m=+0.285839187 container init d2f995aeef1b83922afd350e42bb086710d07ca86cb6101e3309ad300d098f3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T23:04:34.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 podman[29167]: 2024-04-24 23:04:33.731938701 +0000 UTC m=+0.288803755 container start d2f995aeef1b83922afd350e42bb086710d07ca86cb6101e3309ad300d098f3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T23:04:34.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: pidfile_write: ignore empty --pid-file 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: load: jerasure load: lrc 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Git sha 0 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: DB SUMMARY 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: DB Session ID: 2G1GPTIZX6EGIX3DNKOR 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104244 ; 2024-04-24T23:04:34.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.env: 0x55c30367fc60 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.info_log: 0x55c304265aa0 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:04:34.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.statistics: (nil) 2024-04-24T23:04:34.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.use_fsync: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.db_log_dir: 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.wal_dir: 2024-04-24T23:04:34.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.write_buffer_manager: 0x55c3042381e0 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:04:34.034 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.unordered_write: 0 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.row_cache: None 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.wal_filter: None 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.wal_compression: 0 2024-04-24T23:04:34.035 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:04:34.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_open_files: -1 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Compression algorithms supported: 2024-04-24T23:04:34.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: kZSTD supported: 0 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.merge_operator: 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_filter: None 2024-04-24T23:04:34.038 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c304265ba0) 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_type: 0 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_index_type: 0 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_shortening: 1 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: checksum: 4 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: no_block_cache: 0 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache: 0x55c304269090 2024-04-24T23:04:34.039 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_options: 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: capacity : 536870912 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_shard_bits : 4 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: strict_capacity_limit : 0 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_compressed: (nil) 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: persistent_cache: (nil) 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size: 4096 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size_deviation: 10 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_restart_interval: 16 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_block_restart_interval: 1 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: metadata_block_size: 4096 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: partition_filters: 0 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: use_delta_encoding: 1 2024-04-24T23:04:34.040 INFO:journalctl@ceph.mon.a.smithi112.stdout: filter_policy: bloomfilter 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: whole_key_filtering: 1 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: verify_compression: 0 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: format_version: 5 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_index_compression: 1 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_align: 0 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: max_auto_readahead_size: 262144 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: prepopulate_block_cache: 0 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression: NoCompression 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:04:34.041 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.num_levels: 7 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:04:34.042 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:04:34.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:04:34.044 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:04:34.045 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.ttl: 2592000 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:04:34.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 380a4677-ce14-4439-b777-048f0d6a5abe 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999873760832, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999873761745, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99442, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97362, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14050, "raw_average_key_size": 49, "raw_value_size": 89689, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713999873, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "380a4677-ce14-4439-b777-048f0d6a5abe", "db_session_id": "2G1GPTIZX6EGIX3DNKOR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999873761832, "job": 1, "event": "recovery_finished"} 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c30435a000 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: DB pointer 0x55c304344000 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:04:34.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** DB Stats ** 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: L0 2/0 98.95 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Sum 2/0 98.95 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:04:34.048 INFO:journalctl@ceph.mon.a.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 166.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 166.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:04:34.049 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative compaction: 0.00 GB write, 33.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval compaction: 0.00 GB write, 33.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache BinnedLRUCache@0x55c304269090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: starting mon.a rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a@-1(???) e1 preinit fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a@-1(???).mds e1 new map 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a@-1(???).mds e1 print_map 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: e1 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: legacy client fscid: -1 2024-04-24T23:04:34.050 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout: No filesystems configured 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 bash[29167]: d2f995aeef1b83922afd350e42bb086710d07ca86cb6101e3309ad300d098f3d 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 systemd[1]: Started Ceph mon.a for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: monmap epoch 1 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: last_changed 2024-04-24T23:04:25.846546+0000 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: min_mon_release 19 (squid) 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: election_strategy: 1 2024-04-24T23:04:34.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:04:34.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: fsmap 2024-04-24T23:04:34.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:04:34.052 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:33 smithi112 ceph-mon[29186]: mgrmap e1: no daemons active 2024-04-24T23:04:34.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.240+0000 7f40cb8a4700 1 Processor -- start 2024-04-24T23:04:34.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.240+0000 7f40cb8a4700 1 -- start start 2024-04-24T23:04:34.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.240+0000 7f40cb8a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c410ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:34.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.240+0000 7f40cb8a4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40c407bd50 con 0x7f40c4108830 2024-04-24T23:04:34.892 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.240+0000 7f40c9640700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c410ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.240+0000 7f40c9640700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c410ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39480/0 (socket says 172.21.15.112:39480) 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.240+0000 7f40c9640700 1 -- 172.21.15.112:0/3537382178 learned_addr learned my addr 172.21.15.112:0/3537382178 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.241+0000 7f40c9640700 1 -- 172.21.15.112:0/3537382178 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c410b160 con 0x7f40c4108830 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.241+0000 7f40c9640700 1 --2- 172.21.15.112:0/3537382178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c410ac20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f40c00098a0 tx=0x7f40c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903461ab7d215d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.241+0000 7f40b7fff700 1 -- 172.21.15.112:0/3537382178 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c0004160 con 0x7f40c4108830 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.241+0000 7f40b7fff700 1 -- 172.21.15.112:0/3537382178 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f40c0004300 con 0x7f40c4108830 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.241+0000 7f40b7fff700 1 -- 172.21.15.112:0/3537382178 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c0011c00 con 0x7f40c4108830 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 -- 172.21.15.112:0/3537382178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 msgr2=0x7f40c410ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 --2- 172.21.15.112:0/3537382178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c410ac20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f40c00098a0 tx=0x7f40c0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 -- 172.21.15.112:0/3537382178 shutdown_connections 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 --2- 172.21.15.112:0/3537382178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c410ac20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 -- 172.21.15.112:0/3537382178 >> 172.21.15.112:0/3537382178 conn(0x7f40c41021a0 msgr2=0x7f40c41045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:34.893 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 -- 172.21.15.112:0/3537382178 shutdown_connections 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 -- 172.21.15.112:0/3537382178 wait complete. 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 Processor -- start 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 -- start start 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.242+0000 7f40cb8a4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40c410c120 con 0x7f40c4108830 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40c9640700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40c9640700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c419bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39494/0 (socket says 172.21.15.112:39494) 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40c9640700 1 -- 172.21.15.112:0/2404808980 learned_addr learned my addr 172.21.15.112:0/2404808980 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40c9640700 1 -- 172.21.15.112:0/2404808980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c419c240 con 0x7f40c4108830 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40c9640700 1 --2- 172.21.15.112:0/2404808980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c419bd00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f40c0011bd0 tx=0x7f40c0011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40b67fc700 1 -- 172.21.15.112:0/2404808980 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c0003970 con 0x7f40c4108830 2024-04-24T23:04:34.894 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40b67fc700 1 -- 172.21.15.112:0/2404808980 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f40c0003b10 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40b67fc700 1 -- 172.21.15.112:0/2404808980 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40c00245a0 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c419c4d0 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c419f1b0 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40b67fc700 1 -- 172.21.15.112:0/2404808980 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f40c0024be0 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.243+0000 7f40b67fc700 1 -- 172.21.15.112:0/2404808980 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f40c0019700 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.244+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c4063890 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.246+0000 7f40b67fc700 1 -- 172.21.15.112:0/2404808980 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f40c0043dd0 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.313+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f40c419f380 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.317+0000 7f40b67fc700 1 -- 172.21.15.112:0/2404808980 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f40c0032030 con 0x7f40c4108830 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.318+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 msgr2=0x7f40c419bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.318+0000 7f40cb8a4700 1 --2- 172.21.15.112:0/2404808980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c419bd00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f40c0011bd0 tx=0x7f40c0011cb0 comp rx=0 tx=0).stop 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.318+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 shutdown_connections 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.318+0000 7f40cb8a4700 1 --2- 172.21.15.112:0/2404808980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c4108830 0x7f40c419bd00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:34.895 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.318+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 >> 172.21.15.112:0/2404808980 conn(0x7f40c41021a0 msgr2=0x7f40c4102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:34.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.318+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 shutdown_connections 2024-04-24T23:04:34.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.318+0000 7f40cb8a4700 1 -- 172.21.15.112:0/2404808980 wait complete. 2024-04-24T23:04:34.896 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T23:04:34.918 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:34.918 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mgr... 2024-04-24T23:04:34.919 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-24T23:04:34.919 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-24T23:04:35.163 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:04:35.163 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-24T23:04:35.309 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e62ba378-028e-11ef-bc93-c7b262605968.target.wants/ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-e62ba378-028e-11ef-bc93-c7b262605968@.service. 2024-04-24T23:04:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:35 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2404808980' entity='client.admin' 2024-04-24T23:04:35.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:35 smithi112 systemd[1]: Starting Ceph mgr.x for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:04:35.831 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:35 smithi112 podman[29402]: 2024-04-24 23:04:35.536807644 +0000 UTC m=+0.105979012 container create ba5e22e86d481152f58933821f2fce9dc93b47d7977a77bd22fc18d4ee4a8af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T23:04:35.832 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:35 smithi112 podman[29402]: 2024-04-24 23:04:35.4488019 +0000 UTC m=+0.017973239 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:04:35.832 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:35 smithi112 podman[29402]: 2024-04-24 23:04:35.755718451 +0000 UTC m=+0.324889777 container init ba5e22e86d481152f58933821f2fce9dc93b47d7977a77bd22fc18d4ee4a8af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:04:35.832 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:35 smithi112 podman[29402]: 2024-04-24 23:04:35.758217236 +0000 UTC m=+0.327388562 container start ba5e22e86d481152f58933821f2fce9dc93b47d7977a77bd22fc18d4ee4a8af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T23:04:35.832 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:35 smithi112 bash[29402]: ba5e22e86d481152f58933821f2fce9dc93b47d7977a77bd22fc18d4ee4a8af4 2024-04-24T23:04:35.847 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:04:35.847 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-24T23:04:35.855 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:04:35.855 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-24T23:04:35.855 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-24T23:04:35.855 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:04:35.865 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:04:35.865 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-24T23:04:35.874 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:04:35.874 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-24T23:04:35.874 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-24T23:04:35.874 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-24T23:04:35.874 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr to start... 2024-04-24T23:04:35.874 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr... 2024-04-24T23:04:36.090 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:35 smithi112 systemd[1]: Started Ceph mgr.x for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:04:36.090 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:35 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:35.918+0000 7fe9012a3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:04:36.401 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:36.108+0000 7fe9012a3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:04:36.402 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:36.210+0000 7fe9012a3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:04:36.732 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:36 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2996834092' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:04:36.732 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:36.533+0000 7fe9012a3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:04:36.733 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:36.616+0000 7fe9012a3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:04:37.004 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:37.014 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-24T23:04:37.015 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:04:37.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:04:37.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:04:28.857321+0000", 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:37.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.321+0000 7facf8b9c700 1 Processor -- start 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.321+0000 7facf8b9c700 1 -- start start 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.321+0000 7facf8b9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.321+0000 7facf8b9c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facf407c800 con 0x7facf407ddc0 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.321+0000 7facf259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.321+0000 7facf259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf407c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39540/0 (socket says 172.21.15.112:39540) 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.321+0000 7facf259c700 1 -- 172.21.15.112:0/3112488329 learned_addr learned my addr 172.21.15.112:0/3112488329 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.322+0000 7facf259c700 1 -- 172.21.15.112:0/3112488329 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf407c980 con 0x7facf407ddc0 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.322+0000 7facf259c700 1 --2- 172.21.15.112:0/3112488329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf407c2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7face40098a0 tx=0x7face4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f53483c0c1c1bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.322+0000 7facf159a700 1 -- 172.21.15.112:0/3112488329 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4004160 con 0x7facf407ddc0 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.322+0000 7facf159a700 1 -- 172.21.15.112:0/3112488329 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7face4004300 con 0x7facf407ddc0 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.322+0000 7facf159a700 1 -- 172.21.15.112:0/3112488329 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4011c00 con 0x7facf407ddc0 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.323+0000 7facf8b9c700 1 -- 172.21.15.112:0/3112488329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 msgr2=0x7facf407c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.019 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.323+0000 7facf8b9c700 1 --2- 172.21.15.112:0/3112488329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf407c2c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7face40098a0 tx=0x7face4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.323+0000 7facf8b9c700 1 -- 172.21.15.112:0/3112488329 shutdown_connections 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.323+0000 7facf8b9c700 1 --2- 172.21.15.112:0/3112488329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf407c2c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.323+0000 7facf8b9c700 1 -- 172.21.15.112:0/3112488329 >> 172.21.15.112:0/3112488329 conn(0x7facf4103300 msgr2=0x7facf4105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.323+0000 7facf8b9c700 1 -- 172.21.15.112:0/3112488329 shutdown_connections 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.323+0000 7facf8b9c700 1 -- 172.21.15.112:0/3112488329 wait complete. 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.323+0000 7facf8b9c700 1 Processor -- start 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf8b9c700 1 -- start start 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf8b9c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf8b9c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facf410e2d0 con 0x7facf407ddc0 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf419df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39548/0 (socket says 172.21.15.112:39548) 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf259c700 1 -- 172.21.15.112:0/2996834092 learned_addr learned my addr 172.21.15.112:0/2996834092 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf259c700 1 -- 172.21.15.112:0/2996834092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf419e450 con 0x7facf407ddc0 2024-04-24T23:04:37.020 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf259c700 1 --2- 172.21.15.112:0/2996834092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf419df10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7face4009fb0 tx=0x7face4011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7face37fe700 1 -- 172.21.15.112:0/2996834092 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face40038b0 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf419e6e0 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7face37fe700 1 -- 172.21.15.112:0/2996834092 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7face401a430 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7face37fe700 1 -- 172.21.15.112:0/2996834092 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4019630 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.324+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf41a13d0 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.325+0000 7face37fe700 1 -- 172.21.15.112:0/2996834092 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7face4019cc0 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.325+0000 7face37fe700 1 -- 172.21.15.112:0/2996834092 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7face4024730 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.325+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf4063890 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.328+0000 7face37fe700 1 -- 172.21.15.112:0/2996834092 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7face4043dd0 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.400+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7facf41a15a0 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.400+0000 7face37fe700 1 -- 172.21.15.112:0/2996834092 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7face4032020 con 0x7facf407ddc0 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.401+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 msgr2=0x7facf419df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.401+0000 7facf8b9c700 1 --2- 172.21.15.112:0/2996834092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf419df10 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7face4009fb0 tx=0x7face4011c60 comp rx=0 tx=0).stop 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.401+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 shutdown_connections 2024-04-24T23:04:37.021 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.401+0000 7facf8b9c700 1 --2- 172.21.15.112:0/2996834092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facf407ddc0 0x7facf419df10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.401+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 >> 172.21.15.112:0/2996834092 conn(0x7facf4103300 msgr2=0x7facf4103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.402+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 shutdown_connections 2024-04-24T23:04:37.022 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.402+0000 7facf8b9c700 1 -- 172.21.15.112:0/2996834092 wait complete. 2024-04-24T23:04:37.022 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (1/15)... 2024-04-24T23:04:38.141 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:37 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:37.806+0000 7fe9012a3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:04:38.141 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:37 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:37.973+0000 7fe9012a3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:04:38.141 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:38.139+0000 7fe9012a3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:04:38.715 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:38.422+0000 7fe9012a3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:04:38.715 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:38.513+0000 7fe9012a3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:04:38.716 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:38.713+0000 7fe9012a3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:04:39.028 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:38.906+0000 7fe9012a3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:04:39.770 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:39 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:39.522+0000 7fe9012a3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:04:40.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:39 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2746379620' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:04:40.029 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:39 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:39.954+0000 7fe9012a3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:04:40.528 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:40.050+0000 7fe9012a3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:04:40.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:40.254+0000 7fe9012a3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:04:40.874 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:40.776+0000 7fe9012a3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:04:40.874 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:40.867+0000 7fe9012a3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:04:41.240 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:41.107+0000 7fe9012a3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:04:41.240 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:41.238+0000 7fe9012a3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:04:41.531 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:41.332+0000 7fe9012a3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:04:41.531 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:41.446+0000 7fe9012a3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:04:42.036 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:41.536+0000 7fe9012a3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:04:42.613 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:42.341+0000 7fe9012a3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:04:42.613 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:42.558+0000 7fe9012a3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: Activating manager daemon x 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: mgrmap e2: x(active, starting, since 0.00308281s) 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: Manager daemon x is now available 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' 2024-04-24T23:04:43.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:42 smithi112 ceph-mon[29186]: from='mgr.14100 172.21.15.112:0/3679755773' entity='mgr.x' 2024-04-24T23:04:45.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:44 smithi112 ceph-mon[29186]: mgrmap e3: x(active, since 1.00746s) 2024-04-24T23:04:45.913 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:45 smithi112 ceph-mon[29186]: mgrmap e4: x(active, since 2s) 2024-04-24T23:04:45.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:45.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:45.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:04:45.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:04:45.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:04:45.918 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:04:45.919 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:04:45.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:45.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:04:45.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:45.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:04:45.920 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:04:45.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:04:45.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:04:45.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:04:45.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:04:45.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:45.928 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:04:45.929 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:04:45.930 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:04:28.857321+0000", 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.768+0000 7f60e1c6b700 1 Processor -- start 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.768+0000 7f60e1c6b700 1 -- start start 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.769+0000 7f60e1c6b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc10ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.769+0000 7f60e1c6b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc07bdf0 con 0x7f60dc108740 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.769+0000 7f60db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc10ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.769+0000 7f60db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc10ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39564/0 (socket says 172.21.15.112:39564) 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.769+0000 7f60db7fe700 1 -- 172.21.15.112:0/3108762835 learned_addr learned my addr 172.21.15.112:0/3108762835 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.769+0000 7f60db7fe700 1 -- 172.21.15.112:0/3108762835 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc10b070 con 0x7f60dc108740 2024-04-24T23:04:45.931 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.769+0000 7f60db7fe700 1 --2- 172.21.15.112:0/3108762835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc10ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0098a0 tx=0x7f60cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f9fcbb449a802a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:45.932 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.769+0000 7f60da7fc700 1 -- 172.21.15.112:0/3108762835 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60cc004160 con 0x7f60dc108740 2024-04-24T23:04:45.932 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60da7fc700 1 -- 172.21.15.112:0/3108762835 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f60cc004300 con 0x7f60dc108740 2024-04-24T23:04:45.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60da7fc700 1 -- 172.21.15.112:0/3108762835 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60cc011c00 con 0x7f60dc108740 2024-04-24T23:04:45.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60e1c6b700 1 -- 172.21.15.112:0/3108762835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 msgr2=0x7f60dc10ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60e1c6b700 1 --2- 172.21.15.112:0/3108762835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc10ab30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0098a0 tx=0x7f60cc009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60e1c6b700 1 -- 172.21.15.112:0/3108762835 shutdown_connections 2024-04-24T23:04:45.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60e1c6b700 1 --2- 172.21.15.112:0/3108762835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc10ab30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60e1c6b700 1 -- 172.21.15.112:0/3108762835 >> 172.21.15.112:0/3108762835 conn(0x7f60dc102170 msgr2=0x7f60dc104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60e1c6b700 1 -- 172.21.15.112:0/3108762835 shutdown_connections 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.770+0000 7f60e1c6b700 1 -- 172.21.15.112:0/3108762835 wait complete. 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.771+0000 7f60e1c6b700 1 Processor -- start 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.771+0000 7f60e1c6b700 1 -- start start 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.771+0000 7f60e1c6b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.771+0000 7f60e1c6b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc10c0f0 con 0x7f60dc108740 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.771+0000 7f60db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.771+0000 7f60db7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39568/0 (socket says 172.21.15.112:39568) 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.771+0000 7f60db7fe700 1 -- 172.21.15.112:0/2746379620 learned_addr learned my addr 172.21.15.112:0/2746379620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.772+0000 7f60db7fe700 1 -- 172.21.15.112:0/2746379620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc19e3f0 con 0x7f60dc108740 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.772+0000 7f60db7fe700 1 --2- 172.21.15.112:0/2746379620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc19deb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009fb0 tx=0x7f60cc011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.772+0000 7f60d8ff9700 1 -- 172.21.15.112:0/2746379620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60cc0038b0 con 0x7f60dc108740 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.772+0000 7f60d8ff9700 1 -- 172.21.15.112:0/2746379620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f60cc01a430 con 0x7f60dc108740 2024-04-24T23:04:45.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.772+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc19e680 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.772+0000 7f60d8ff9700 1 -- 172.21.15.112:0/2746379620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60cc019630 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.772+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc1a1370 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.773+0000 7f60d8ff9700 1 -- 172.21.15.112:0/2746379620 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f60cc019930 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.773+0000 7f60d8ff9700 1 -- 172.21.15.112:0/2746379620 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f60cc0243f0 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.773+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc063890 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.775+0000 7f60d8ff9700 1 -- 172.21.15.112:0/2746379620 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f60cc019db0 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.848+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f60dc1a1540 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.848+0000 7f60d8ff9700 1 -- 172.21.15.112:0/2746379620 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f60cc01a5e0 con 0x7f60dc108740 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.849+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 msgr2=0x7f60dc19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.849+0000 7f60e1c6b700 1 --2- 172.21.15.112:0/2746379620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc19deb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009fb0 tx=0x7f60cc011c60 comp rx=0 tx=0).stop 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.849+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 shutdown_connections 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.849+0000 7f60e1c6b700 1 --2- 172.21.15.112:0/2746379620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60dc108740 0x7f60dc19deb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.849+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 >> 172.21.15.112:0/2746379620 conn(0x7f60dc102170 msgr2=0x7f60dc102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:45.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.849+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 shutdown_connections 2024-04-24T23:04:45.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.849+0000 7f60e1c6b700 1 -- 172.21.15.112:0/2746379620 wait complete. 2024-04-24T23:04:45.936 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (2/15)... 2024-04-24T23:04:50.913 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2334829141' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:04:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:04:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:04:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:04:51.193 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:51.194 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-24T23:04:51.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:04:51.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:51.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:04:51.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:51.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:04:51.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:04:51.195 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:04:51.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:04:51.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:04:51.204 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:04:51.205 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:51.206 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:04:28.857321+0000", 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.313+0000 7f1c26112700 1 Processor -- start 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.313+0000 7f1c26112700 1 -- start start 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.313+0000 7f1c26112700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c20108570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.313+0000 7f1c26112700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20108b40 con 0x7f1c20108150 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.313+0000 7f1c1f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c20108570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.313+0000 7f1c1f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c20108570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50840/0 (socket says 172.21.15.112:50840) 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.314+0000 7f1c1f7fe700 1 -- 172.21.15.112:0/4060379062 learned_addr learned my addr 172.21.15.112:0/4060379062 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.314+0000 7f1c1f7fe700 1 -- 172.21.15.112:0/4060379062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c20106db0 con 0x7f1c20108150 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.314+0000 7f1c1f7fe700 1 --2- 172.21.15.112:0/4060379062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c20108570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c100098a0 tx=0x7f1c10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375678613ea88f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.314+0000 7f1c1e7fc700 1 -- 172.21.15.112:0/4060379062 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c10004160 con 0x7f1c20108150 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.314+0000 7f1c1e7fc700 1 -- 172.21.15.112:0/4060379062 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1c10004300 con 0x7f1c20108150 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.314+0000 7f1c1e7fc700 1 -- 172.21.15.112:0/4060379062 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c10011c00 con 0x7f1c20108150 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.315+0000 7f1c26112700 1 -- 172.21.15.112:0/4060379062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 msgr2=0x7f1c20108570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.315+0000 7f1c26112700 1 --2- 172.21.15.112:0/4060379062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c20108570 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c100098a0 tx=0x7f1c10009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.315+0000 7f1c26112700 1 -- 172.21.15.112:0/4060379062 shutdown_connections 2024-04-24T23:04:51.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.315+0000 7f1c26112700 1 --2- 172.21.15.112:0/4060379062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c20108570 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.315+0000 7f1c26112700 1 -- 172.21.15.112:0/4060379062 >> 172.21.15.112:0/4060379062 conn(0x7f1c20101900 msgr2=0x7f1c20103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.315+0000 7f1c26112700 1 -- 172.21.15.112:0/4060379062 shutdown_connections 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.315+0000 7f1c26112700 1 -- 172.21.15.112:0/4060379062 wait complete. 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.315+0000 7f1c26112700 1 Processor -- start 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c26112700 1 -- start start 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c26112700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c2019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c26112700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010b170 con 0x7f1c20108150 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c1f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c2019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c1f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c2019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50844/0 (socket says 172.21.15.112:50844) 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c1f7fe700 1 -- 172.21.15.112:0/2334829141 learned_addr learned my addr 172.21.15.112:0/2334829141 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c1f7fe700 1 -- 172.21.15.112:0/2334829141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2019b090 con 0x7f1c20108150 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c1f7fe700 1 --2- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c2019ab50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c10003fb0 tx=0x7f1c10011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:51.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.316+0000 7f1c1cff9700 1 -- 172.21.15.112:0/2334829141 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c100039c0 con 0x7f1c20108150 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.317+0000 7f1c1cff9700 1 -- 172.21.15.112:0/2334829141 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1c10003b60 con 0x7f1c20108150 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.317+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2019b320 con 0x7f1c20108150 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.317+0000 7f1c1cff9700 1 -- 172.21.15.112:0/2334829141 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c1001a6b0 con 0x7f1c20108150 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.317+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2019e010 con 0x7f1c20108150 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.317+0000 7f1c1cff9700 1 -- 172.21.15.112:0/2334829141 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1c1001aa30 con 0x7f1c20108150 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.318+0000 7f1c1cff9700 1 --2- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1c08039000 0x7f1c0803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.318+0000 7f1c1cff9700 1 -- 172.21.15.112:0/2334829141 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1c1004e4f0 con 0x7f1c20108150 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.318+0000 7f1c1effd700 1 --2- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1c08039000 0x7f1c0803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.318+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c20063890 con 0x7f1c20108150 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.318+0000 7f1c1effd700 1 --2- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1c08039000 0x7f1c0803b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c14006fd0 tx=0x7f1c14006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:51.210 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.323+0000 7f1c1cff9700 1 -- 172.21.15.112:0/2334829141 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c10003d10 con 0x7f1c20108150 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.601+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1c2019e1e0 con 0x7f1c20108150 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.602+0000 7f1c1cff9700 1 -- 172.21.15.112:0/2334829141 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f1c1004df10 con 0x7f1c20108150 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1c08039000 msgr2=0x7f1c0803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 --2- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1c08039000 0x7f1c0803b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c14006fd0 tx=0x7f1c14006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 msgr2=0x7f1c2019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 --2- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c2019ab50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c10003fb0 tx=0x7f1c10011bd0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 shutdown_connections 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 --2- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1c08039000 0x7f1c0803b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 --2- 172.21.15.112:0/2334829141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c20108150 0x7f1c2019ab50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 >> 172.21.15.112:0/2334829141 conn(0x7f1c20101900 msgr2=0x7f1c20102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:51.211 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 shutdown_connections 2024-04-24T23:04:51.212 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:50.604+0000 7f1c26112700 1 -- 172.21.15.112:0/2334829141 wait complete. 2024-04-24T23:04:51.212 INFO:teuthology.orchestra.run.smithi112.stdout:mgr is available 2024-04-24T23:04:52.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:51 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2960783896' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:04:52.823 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:52.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:04:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:04:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:04:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:04:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:04:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:04:52.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.570+0000 7fee05195700 1 Processor -- start 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.570+0000 7fee05195700 1 -- start start 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.570+0000 7fee05195700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee00106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.570+0000 7fee05195700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee00107590 con 0x7fee00106ba0 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.570+0000 7fedfed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee00106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.834 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.571+0000 7fedfed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee00106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52230/0 (socket says 172.21.15.112:52230) 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.571+0000 7fedfed9d700 1 -- 172.21.15.112:0/3624654392 learned_addr learned my addr 172.21.15.112:0/3624654392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.571+0000 7fedfed9d700 1 -- 172.21.15.112:0/3624654392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee00107da0 con 0x7fee00106ba0 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.571+0000 7fedfed9d700 1 --2- 172.21.15.112:0/3624654392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee00106fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fedf00098a0 tx=0x7fedf0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18558bafcf35cb7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.571+0000 7fedfdd9b700 1 -- 172.21.15.112:0/3624654392 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedf0004160 con 0x7fee00106ba0 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.571+0000 7fedfdd9b700 1 -- 172.21.15.112:0/3624654392 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fedf0004300 con 0x7fee00106ba0 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.571+0000 7fedfdd9b700 1 -- 172.21.15.112:0/3624654392 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedf0011c00 con 0x7fee00106ba0 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.571+0000 7fee05195700 1 -- 172.21.15.112:0/3624654392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 msgr2=0x7fee00106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.572+0000 7fee05195700 1 --2- 172.21.15.112:0/3624654392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee00106fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fedf00098a0 tx=0x7fedf0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.572+0000 7fee05195700 1 -- 172.21.15.112:0/3624654392 shutdown_connections 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.572+0000 7fee05195700 1 --2- 172.21.15.112:0/3624654392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee00106fc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.835 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.572+0000 7fee05195700 1 -- 172.21.15.112:0/3624654392 >> 172.21.15.112:0/3624654392 conn(0x7fee00102160 msgr2=0x7fee00104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.572+0000 7fee05195700 1 -- 172.21.15.112:0/3624654392 shutdown_connections 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.572+0000 7fee05195700 1 -- 172.21.15.112:0/3624654392 wait complete. 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.572+0000 7fee05195700 1 Processor -- start 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fee05195700 1 -- start start 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fee05195700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee0007ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fee05195700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee0010b150 con 0x7fee00106ba0 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fedfed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee0007ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fedfed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee0007ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52242/0 (socket says 172.21.15.112:52242) 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fedfed9d700 1 -- 172.21.15.112:0/2960783896 learned_addr learned my addr 172.21.15.112:0/2960783896 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fedfed9d700 1 -- 172.21.15.112:0/2960783896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0019ea70 con 0x7fee00106ba0 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fedfed9d700 1 --2- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee0007ba30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fedf00118c0 tx=0x7fedf0011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fede7fff700 1 -- 172.21.15.112:0/2960783896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedf0003af0 con 0x7fee00106ba0 2024-04-24T23:04:52.836 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0019ec40 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fede7fff700 1 -- 172.21.15.112:0/2960783896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fedf0020070 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.573+0000 7fede7fff700 1 -- 172.21.15.112:0/2960783896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedf00195e0 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.574+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0019f060 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.574+0000 7fede7fff700 1 -- 172.21.15.112:0/2960783896 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fedf00198a0 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.574+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0010b2d0 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.575+0000 7fede7fff700 1 --2- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fede8039000 0x7fede803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.575+0000 7fede7fff700 1 -- 172.21.15.112:0/2960783896 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fedf004d2a0 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.575+0000 7fedfe59c700 1 --2- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fede8039000 0x7fede803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.575+0000 7fedfe59c700 1 --2- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fede8039000 0x7fede803b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fedf4006fd0 tx=0x7fedf4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.580+0000 7fede7fff700 1 -- 172.21.15.112:0/2960783896 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fedf0023530 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.776+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fee001a0a50 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.777+0000 7fede7fff700 1 -- 172.21.15.112:0/2960783896 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7fedf0003c90 con 0x7fee00106ba0 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fede8039000 msgr2=0x7fede803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.837 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 --2- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fede8039000 0x7fede803b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fedf4006fd0 tx=0x7fedf4006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 msgr2=0x7fee0007ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 --2- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee0007ba30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fedf00118c0 tx=0x7fedf0011c80 comp rx=0 tx=0).stop 2024-04-24T23:04:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 shutdown_connections 2024-04-24T23:04:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 --2- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fede8039000 0x7fede803b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 --2- 172.21.15.112:0/2960783896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee00106ba0 0x7fee0007ba30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 >> 172.21.15.112:0/2960783896 conn(0x7fee00102160 msgr2=0x7fee00102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 shutdown_connections 2024-04-24T23:04:52.838 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.779+0000 7fee05195700 1 -- 172.21.15.112:0/2960783896 wait complete. 2024-04-24T23:04:52.839 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling cephadm module... 2024-04-24T23:04:53.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3807466429' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T23:04:53.779 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:53 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: ignoring --setuser ceph since I am not root 2024-04-24T23:04:53.779 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:53 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: ignoring --setgroup ceph since I am not root 2024-04-24T23:04:53.779 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:53 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:53.725+0000 7fc1fbdf2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:04:54.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.248+0000 7f28d6efa700 1 Processor -- start 2024-04-24T23:04:54.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.248+0000 7f28d6efa700 1 -- start start 2024-04-24T23:04:54.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.249+0000 7f28d6efa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d01074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.249+0000 7f28d6efa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0107a70 con 0x7f28d0107080 2024-04-24T23:04:54.142 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.249+0000 7f28d4c96700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d01074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.249+0000 7f28d4c96700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d01074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52254/0 (socket says 172.21.15.112:52254) 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.249+0000 7f28d4c96700 1 -- 172.21.15.112:0/1179947671 learned_addr learned my addr 172.21.15.112:0/1179947671 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.249+0000 7f28d4c96700 1 -- 172.21.15.112:0/1179947671 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d0107da0 con 0x7f28d0107080 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.249+0000 7f28d4c96700 1 --2- 172.21.15.112:0/1179947671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d01074a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f28c40098a0 tx=0x7f28c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d35e32864e6b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28cf7fe700 1 -- 172.21.15.112:0/1179947671 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c4004160 con 0x7f28d0107080 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28cf7fe700 1 -- 172.21.15.112:0/1179947671 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f28c4004300 con 0x7f28d0107080 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28cf7fe700 1 -- 172.21.15.112:0/1179947671 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c4011c00 con 0x7f28d0107080 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28d6efa700 1 -- 172.21.15.112:0/1179947671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 msgr2=0x7f28d01074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28d6efa700 1 --2- 172.21.15.112:0/1179947671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d01074a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f28c40098a0 tx=0x7f28c4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28d6efa700 1 -- 172.21.15.112:0/1179947671 shutdown_connections 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28d6efa700 1 --2- 172.21.15.112:0/1179947671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d01074a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28d6efa700 1 -- 172.21.15.112:0/1179947671 >> 172.21.15.112:0/1179947671 conn(0x7f28d01018c0 msgr2=0x7f28d0103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28d6efa700 1 -- 172.21.15.112:0/1179947671 shutdown_connections 2024-04-24T23:04:54.143 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.250+0000 7f28d6efa700 1 -- 172.21.15.112:0/1179947671 wait complete. 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.251+0000 7f28d6efa700 1 Processor -- start 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.251+0000 7f28d6efa700 1 -- start start 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.251+0000 7f28d6efa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.251+0000 7f28d6efa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f28d010b150 con 0x7f28d0107080 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.251+0000 7f28d4c96700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.251+0000 7f28d4c96700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52264/0 (socket says 172.21.15.112:52264) 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.251+0000 7f28d4c96700 1 -- 172.21.15.112:0/3807466429 learned_addr learned my addr 172.21.15.112:0/3807466429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.251+0000 7f28d4c96700 1 -- 172.21.15.112:0/3807466429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d019b110 con 0x7f28d0107080 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.252+0000 7f28d4c96700 1 --2- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d019abd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f28c4011bd0 tx=0x7f28c4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.252+0000 7f28cdffb700 1 -- 172.21.15.112:0/3807466429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c4003970 con 0x7f28d0107080 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.252+0000 7f28cdffb700 1 -- 172.21.15.112:0/3807466429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f28c4003b10 con 0x7f28d0107080 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.252+0000 7f28cdffb700 1 -- 172.21.15.112:0/3807466429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c40245a0 con 0x7f28d0107080 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.252+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d019b3a0 con 0x7f28d0107080 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.252+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d019e090 con 0x7f28d0107080 2024-04-24T23:04:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.253+0000 7f28cdffb700 1 -- 172.21.15.112:0/3807466429 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f28c401a9a0 con 0x7f28d0107080 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.253+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d0063890 con 0x7f28d0107080 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.253+0000 7f28cdffb700 1 --2- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f28c0039000 0x7f28c003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.253+0000 7f28cdffb700 1 -- 172.21.15.112:0/3807466429 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f28c404d170 con 0x7f28d0107080 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.253+0000 7f28cffff700 1 --2- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f28c0039000 0x7f28c003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.254+0000 7f28cffff700 1 --2- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f28c0039000 0x7f28c003b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006fd0 tx=0x7f28bc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.258+0000 7f28cdffb700 1 -- 172.21.15.112:0/3807466429 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f28c402ce10 con 0x7f28d0107080 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.508+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f28d019e260 con 0x7f28d0107080 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.559+0000 7f28cdffb700 1 -- 172.21.15.112:0/3807466429 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f28d019e260 con 0x7f28d0107080 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.561+0000 7f28cdffb700 1 -- 172.21.15.112:0/3807466429 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f28c401a430 con 0x7f28d0107080 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f28c0039000 msgr2=0x7f28c003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 --2- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f28c0039000 0x7f28c003b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006fd0 tx=0x7f28bc006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 msgr2=0x7f28d019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 --2- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d019abd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f28c4011bd0 tx=0x7f28c4011cb0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 shutdown_connections 2024-04-24T23:04:54.145 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 --2- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f28c0039000 0x7f28c003b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 --2- 172.21.15.112:0/3807466429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28d0107080 0x7f28d019abd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 >> 172.21.15.112:0/3807466429 conn(0x7f28d01018c0 msgr2=0x7f28d0102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:54.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 shutdown_connections 2024-04-24T23:04:54.146 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.562+0000 7f28d6efa700 1 -- 172.21.15.112:0/3807466429 wait complete. 2024-04-24T23:04:54.261 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:53 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:53.907+0000 7fc1fbdf2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:04:54.261 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:54 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:54.009+0000 7fc1fbdf2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:04:54.528 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:54 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:54.321+0000 7fc1fbdf2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:04:54.528 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:54 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:54.407+0000 7fc1fbdf2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:04:54.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:54 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3807466429' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T23:04:54.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:54 smithi112 ceph-mon[29186]: mgrmap e5: x(active, since 11s) 2024-04-24T23:04:55.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:55.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T23:04:55.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:04:55.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:04:55.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:04:55.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:55.508 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.551+0000 7f1e1ce48700 1 Processor -- start 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.551+0000 7f1e1ce48700 1 -- start start 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.551+0000 7f1e1ce48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e18108100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.551+0000 7f1e1ce48700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181086d0 con 0x7f1e18107ce0 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.551+0000 7f1e1659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e18108100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.551+0000 7f1e1659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e18108100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52282/0 (socket says 172.21.15.112:52282) 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.551+0000 7f1e1659c700 1 -- 172.21.15.112:0/991436619 learned_addr learned my addr 172.21.15.112:0/991436619 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.552+0000 7f1e1659c700 1 -- 172.21.15.112:0/991436619 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e18108e90 con 0x7f1e18107ce0 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.552+0000 7f1e1659c700 1 --2- 172.21.15.112:0/991436619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e18108100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1e080098a0 tx=0x7f1e08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6685f3e42e87dd50 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.552+0000 7f1e1559a700 1 -- 172.21.15.112:0/991436619 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e08004160 con 0x7f1e18107ce0 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.552+0000 7f1e1559a700 1 -- 172.21.15.112:0/991436619 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1e08004300 con 0x7f1e18107ce0 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.552+0000 7f1e1559a700 1 -- 172.21.15.112:0/991436619 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e08011c00 con 0x7f1e18107ce0 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.552+0000 7f1e1ce48700 1 -- 172.21.15.112:0/991436619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 msgr2=0x7f1e18108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.552+0000 7f1e1ce48700 1 --2- 172.21.15.112:0/991436619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e18108100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1e080098a0 tx=0x7f1e08009bb0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.553+0000 7f1e1ce48700 1 -- 172.21.15.112:0/991436619 shutdown_connections 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.553+0000 7f1e1ce48700 1 --2- 172.21.15.112:0/991436619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e18108100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.553+0000 7f1e1ce48700 1 -- 172.21.15.112:0/991436619 >> 172.21.15.112:0/991436619 conn(0x7f1e181032c0 msgr2=0x7f1e181056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.553+0000 7f1e1ce48700 1 -- 172.21.15.112:0/991436619 shutdown_connections 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.553+0000 7f1e1ce48700 1 -- 172.21.15.112:0/991436619 wait complete. 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.553+0000 7f1e1ce48700 1 Processor -- start 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.553+0000 7f1e1ce48700 1 -- start start 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1ce48700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e1807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1ce48700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1810c240 con 0x7f1e18107ce0 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e1807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e1807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52288/0 (socket says 172.21.15.112:52288) 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1659c700 1 -- 172.21.15.112:0/241764175 learned_addr learned my addr 172.21.15.112:0/241764175 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1659c700 1 -- 172.21.15.112:0/241764175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1807e030 con 0x7f1e18107ce0 2024-04-24T23:04:55.510 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1659c700 1 --2- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e1807daf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1e080118c0 tx=0x7f1e08011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e077fe700 1 -- 172.21.15.112:0/241764175 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e08003af0 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e077fe700 1 -- 172.21.15.112:0/241764175 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1e08020070 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e077fe700 1 -- 172.21.15.112:0/241764175 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e080195e0 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1807c220 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.554+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1807c760 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.555+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e1810c3c0 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.556+0000 7f1e077fe700 1 -- 172.21.15.112:0/241764175 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f1e0801a9a0 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.556+0000 7f1e077fe700 1 --2- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1e000390a0 0x7f1e0003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.556+0000 7f1e077fe700 1 -- 172.21.15.112:0/241764175 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e0804d610 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.556+0000 7f1e15d9b700 1 -- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1e000390a0 msgr2=0x7f1e0003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3348191830 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.556+0000 7f1e15d9b700 1 --2- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1e000390a0 0x7f1e0003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.561+0000 7f1e077fe700 1 -- 172.21.15.112:0/241764175 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1e08003c90 con 0x7f1e18107ce0 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.757+0000 7f1e15d9b700 1 -- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1e000390a0 msgr2=0x7f1e0003b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3348191830 2024-04-24T23:04:55.511 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.757+0000 7f1e15d9b700 1 --2- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1e000390a0 0x7f1e0003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.853+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1e1807ca50 con 0x7f1e18107ce0 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.853+0000 7f1e077fe700 1 -- 172.21.15.112:0/241764175 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f1e0804c0d0 con 0x7f1e18107ce0 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.855+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1e000390a0 msgr2=0x7f1e0003b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.855+0000 7f1e1ce48700 1 --2- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1e000390a0 0x7f1e0003b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.855+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 msgr2=0x7f1e1807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.855+0000 7f1e1ce48700 1 --2- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e1807daf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1e080118c0 tx=0x7f1e08011c80 comp rx=0 tx=0).stop 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.855+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 shutdown_connections 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.855+0000 7f1e1ce48700 1 --2- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7f1e000390a0 0x7f1e0003b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.855+0000 7f1e1ce48700 1 --2- 172.21.15.112:0/241764175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e18107ce0 0x7f1e1807daf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.855+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 >> 172.21.15.112:0/241764175 conn(0x7f1e181032c0 msgr2=0x7f1e18103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.856+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 shutdown_connections 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.856+0000 7f1e1ce48700 1 -- 172.21.15.112:0/241764175 wait complete. 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-04-24T23:04:55.512 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 5... 2024-04-24T23:04:55.750 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:04:55 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/241764175' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:04:55.750 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:55 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:55.620+0000 7fc1fbdf2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:04:56.028 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:55 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:55.794+0000 7fc1fbdf2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:04:56.029 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:55 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:55.969+0000 7fc1fbdf2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:04:56.528 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:56 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:56.255+0000 7fc1fbdf2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:04:56.528 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:56 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:56.349+0000 7fc1fbdf2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:04:56.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:56 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:56.565+0000 7fc1fbdf2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:04:56.975 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:56 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:56.753+0000 7fc1fbdf2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:04:57.778 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:57 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:57.350+0000 7fc1fbdf2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:04:58.079 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:57 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:57.781+0000 7fc1fbdf2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:04:58.079 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:57 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:57.877+0000 7fc1fbdf2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:04:58.079 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:58 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:58.077+0000 7fc1fbdf2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:04:58.928 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:58 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:58.594+0000 7fc1fbdf2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:04:58.929 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:58 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:58.685+0000 7fc1fbdf2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:04:59.272 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:58 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:58.927+0000 7fc1fbdf2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:04:59.272 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:59 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:59.060+0000 7fc1fbdf2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:04:59.272 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:59 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:59.155+0000 7fc1fbdf2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:04:59.272 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:59 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:59.270+0000 7fc1fbdf2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:04:59.528 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:04:59 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:04:59.361+0000 7fc1fbdf2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:05:00.430 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:00.161+0000 7fc1fbdf2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:05:00.430 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:00.374+0000 7fc1fbdf2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: Active manager daemon x restarted 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: Activating manager daemon x 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: mgrmap e6: x(active, starting, since 0.00378281s) 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: Manager daemon x is now available 2024-04-24T23:05:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:00.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:00.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:00.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:00.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:05:00.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:00 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:05:01.637 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:01 smithi112 ceph-mon[29186]: Found migration_current of "None". Setting to last migration. 2024-04-24T23:05:01.637 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:01 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:01.637 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:01 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:01.637 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:01 smithi112 ceph-mon[29186]: mgrmap e7: x(active, since 1.00602s) 2024-04-24T23:05:01.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:05:01.922 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.942+0000 7fece0960700 1 Processor -- start 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.942+0000 7fece0960700 1 -- start start 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.942+0000 7fece0960700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc1096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.942+0000 7fece0960700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc109c30 con 0x7fecdc1092d0 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.943+0000 7fecda59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc1096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.943+0000 7fecda59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc1096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52290/0 (socket says 172.21.15.112:52290) 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.943+0000 7fecda59c700 1 -- 172.21.15.112:0/301245422 learned_addr learned my addr 172.21.15.112:0/301245422 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.943+0000 7fecda59c700 1 -- 172.21.15.112:0/301245422 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc1079a0 con 0x7fecdc1092d0 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.943+0000 7fecda59c700 1 --2- 172.21.15.112:0/301245422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc1096f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feccc009e80 tx=0x7feccc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0561a3ba873eee server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.943+0000 7fecd959a700 1 -- 172.21.15.112:0/301245422 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feccc017070 con 0x7fecdc1092d0 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.943+0000 7fecd959a700 1 -- 172.21.15.112:0/301245422 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7feccc00b8c0 con 0x7fecdc1092d0 2024-04-24T23:05:01.923 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.943+0000 7fecd959a700 1 -- 172.21.15.112:0/301245422 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feccc004620 con 0x7fecdc1092d0 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.944+0000 7fece0960700 1 -- 172.21.15.112:0/301245422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 msgr2=0x7fecdc1096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.944+0000 7fece0960700 1 --2- 172.21.15.112:0/301245422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc1096f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feccc009e80 tx=0x7feccc00b330 comp rx=0 tx=0).stop 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.944+0000 7fece0960700 1 -- 172.21.15.112:0/301245422 shutdown_connections 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.944+0000 7fece0960700 1 --2- 172.21.15.112:0/301245422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc1096f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.944+0000 7fece0960700 1 -- 172.21.15.112:0/301245422 >> 172.21.15.112:0/301245422 conn(0x7fecdc103340 msgr2=0x7fecdc105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.944+0000 7fece0960700 1 -- 172.21.15.112:0/301245422 shutdown_connections 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.944+0000 7fece0960700 1 -- 172.21.15.112:0/301245422 wait complete. 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.945+0000 7fece0960700 1 Processor -- start 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.945+0000 7fece0960700 1 -- start start 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.945+0000 7fece0960700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.945+0000 7fece0960700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fecdc10c2d0 con 0x7fecdc1092d0 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.945+0000 7fecda59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.945+0000 7fecda59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc19bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52296/0 (socket says 172.21.15.112:52296) 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.945+0000 7fecda59c700 1 -- 172.21.15.112:0/2869100483 learned_addr learned my addr 172.21.15.112:0/2869100483 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:01.924 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.945+0000 7fecda59c700 1 -- 172.21.15.112:0/2869100483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecdc19c300 con 0x7fecdc1092d0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.946+0000 7fecda59c700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc19bdc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feccc009fd0 tx=0x7feccc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.946+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feccc021020 con 0x7fecdc1092d0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.946+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7feccc00bde0 con 0x7fecdc1092d0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.946+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecdc19c590 con 0x7fecdc1092d0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.946+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feccc017040 con 0x7fecdc1092d0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.946+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecdc19f280 con 0x7fecdc1092d0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.947+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7feccc024020 con 0x7fecdc1092d0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.947+0000 7feccb7fe700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 0x7fecc403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.947+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 --> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fecdc063890 con 0x7fecc4039050 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.947+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feccc00f050 con 0x7fecdc1092d0 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.947+0000 7fecd9d9b700 1 -- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 msgr2=0x7fecc403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3348191830 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:55.947+0000 7fecd9d9b700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 0x7fecc403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:56.148+0000 7fecd9d9b700 1 -- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 msgr2=0x7fecc403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3348191830 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:56.148+0000 7fecd9d9b700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 0x7fecc403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:05:01.925 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:56.549+0000 7fecd9d9b700 1 -- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 msgr2=0x7fecc403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3348191830 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:56.549+0000 7fecd9d9b700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 0x7fecc403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:57.350+0000 7fecd9d9b700 1 -- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 msgr2=0x7fecc403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3348191830 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:57.350+0000 7fecd9d9b700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 0x7fecc403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:58.951+0000 7fecd9d9b700 1 -- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 msgr2=0x7fecc403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3348191830 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:58.951+0000 7fecd9d9b700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 0x7fecc403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:00.378+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7feccc026590 con 0x7fecdc1092d0 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:00.378+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 msgr2=0x7fecc403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:00.378+0000 7feccb7fe700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 0x7fecc403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.379+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7feccc04ee60 con 0x7fecdc1092d0 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.379+0000 7feccb7fe700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fecc403c7e0 0x7fecc403ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.380+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feccc00aea0 con 0x7fecc403c7e0 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.380+0000 7fecd9d9b700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fecc403c7e0 0x7fecc403ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:01.926 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.380+0000 7fecd9d9b700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fecc403c7e0 0x7fecc403ebd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fecd0003a10 tx=0x7fecd0009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.381+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mgr.14118 v2:172.21.15.112:6800/2319978079 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7feccc00aea0 con 0x7fecc403c7e0 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.387+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fecdc10c450 con 0x7fecc403c7e0 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.387+0000 7feccb7fe700 1 -- 172.21.15.112:0/2869100483 <== mgr.14118 v2:172.21.15.112:6800/2319978079 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fecdc10c450 con 0x7fecc403c7e0 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fecc403c7e0 msgr2=0x7fecc403ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fecc403c7e0 0x7fecc403ebd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fecd0003a10 tx=0x7fecd0009290 comp rx=0 tx=0).stop 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 msgr2=0x7fecdc19bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc19bdc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feccc009fd0 tx=0x7feccc003d00 comp rx=0 tx=0).stop 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 shutdown_connections 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fecc403c7e0 0x7fecc403ebd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecdc1092d0 0x7fecdc19bdc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 --2- 172.21.15.112:0/2869100483 >> [v2:172.21.15.112:6800/3348191830,v1:172.21.15.112:6801/3348191830] conn(0x7fecc4039050 0x7fecc403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 >> 172.21.15.112:0/2869100483 conn(0x7fecdc103340 msgr2=0x7fecdc103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 shutdown_connections 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:01.388+0000 7fece0960700 1 -- 172.21.15.112:0/2869100483 wait complete. 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 5 is available 2024-04-24T23:05:01.927 INFO:teuthology.orchestra.run.smithi112.stdout:Setting orchestrator backend to cephadm... 2024-04-24T23:05:02.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:02 smithi112 ceph-mon[29186]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:05:02.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:02 smithi112 ceph-mon[29186]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:05:02.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:02 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:03.095 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.347+0000 7f9666534700 1 Processor -- start 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.347+0000 7f9666534700 1 -- start start 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.347+0000 7f9666534700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f96601080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.347+0000 7f9666534700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96601086b0 con 0x7f9660107cc0 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.347+0000 7f965ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f96601080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.347+0000 7f965ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f96601080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35916/0 (socket says 172.21.15.112:35916) 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.347+0000 7f965ffff700 1 -- 172.21.15.112:0/1506316231 learned_addr learned my addr 172.21.15.112:0/1506316231 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.347+0000 7f965ffff700 1 -- 172.21.15.112:0/1506316231 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9660108ec0 con 0x7f9660107cc0 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f965ffff700 1 --2- 172.21.15.112:0/1506316231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f96601080e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96500098a0 tx=0x7f9650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f559240f895443c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f965effd700 1 -- 172.21.15.112:0/1506316231 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9650004160 con 0x7f9660107cc0 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f965effd700 1 -- 172.21.15.112:0/1506316231 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f9650004300 con 0x7f9660107cc0 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f965effd700 1 -- 172.21.15.112:0/1506316231 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9650011c00 con 0x7f9660107cc0 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f9666534700 1 -- 172.21.15.112:0/1506316231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 msgr2=0x7f96601080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.096 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f9666534700 1 --2- 172.21.15.112:0/1506316231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f96601080e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96500098a0 tx=0x7f9650009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f9666534700 1 -- 172.21.15.112:0/1506316231 shutdown_connections 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f9666534700 1 --2- 172.21.15.112:0/1506316231 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f96601080e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f9666534700 1 -- 172.21.15.112:0/1506316231 >> 172.21.15.112:0/1506316231 conn(0x7f96601032a0 msgr2=0x7f96601056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.348+0000 7f9666534700 1 -- 172.21.15.112:0/1506316231 shutdown_connections 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.349+0000 7f9666534700 1 -- 172.21.15.112:0/1506316231 wait complete. 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.349+0000 7f9666534700 1 Processor -- start 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.349+0000 7f9666534700 1 -- start start 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.349+0000 7f9666534700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f966019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.349+0000 7f9666534700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f966010c270 con 0x7f9660107cc0 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.349+0000 7f965ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f966019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f965ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f966019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35930/0 (socket says 172.21.15.112:35930) 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f965ffff700 1 -- 172.21.15.112:0/2949496259 learned_addr learned my addr 172.21.15.112:0/2949496259 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f965ffff700 1 -- 172.21.15.112:0/2949496259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966019c290 con 0x7f9660107cc0 2024-04-24T23:05:03.097 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f965ffff700 1 --2- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f966019bd50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9650009fb0 tx=0x7f9650011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f965d7fa700 1 -- 172.21.15.112:0/2949496259 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96500038b0 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f965d7fa700 1 -- 172.21.15.112:0/2949496259 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f965001a430 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966019c520 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f965d7fa700 1 -- 172.21.15.112:0/2949496259 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96500195e0 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.350+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966019f210 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.351+0000 7f965d7fa700 1 -- 172.21.15.112:0/2949496259 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f96500198b0 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.351+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f966010c3f0 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.351+0000 7f965d7fa700 1 --2- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f9648038f40 0x7f964803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.351+0000 7f965d7fa700 1 -- 172.21.15.112:0/2949496259 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f965004d330 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.351+0000 7f965f7fe700 1 --2- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f9648038f40 0x7f964803b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.352+0000 7f965f7fe700 1 --2- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f9648038f40 0x7f964803b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9654006fd0 tx=0x7f9654006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.356+0000 7f965d7fa700 1 -- 172.21.15.112:0/2949496259 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9650023c90 con 0x7f9660107cc0 2024-04-24T23:05:03.098 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.434+0000 7f965d7fa700 1 -- 172.21.15.112:0/2949496259 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f965002d960 con 0x7f9660107cc0 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.571+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f96601044f0 con 0x7f9648038f40 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.577+0000 7f965d7fa700 1 -- 172.21.15.112:0/2949496259 <== mgr.14118 v2:172.21.15.112:6800/2319978079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f96601044f0 con 0x7f9648038f40 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f9648038f40 msgr2=0x7f964803b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 --2- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f9648038f40 0x7f964803b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9654006fd0 tx=0x7f9654006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 msgr2=0x7f966019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 --2- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f966019bd50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9650009fb0 tx=0x7f9650011c60 comp rx=0 tx=0).stop 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 shutdown_connections 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 --2- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f9648038f40 0x7f964803b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 --2- 172.21.15.112:0/2949496259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9660107cc0 0x7f966019bd50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 >> 172.21.15.112:0/2949496259 conn(0x7f96601032a0 msgr2=0x7f9660103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 shutdown_connections 2024-04-24T23:05:03.099 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:02.579+0000 7f9666534700 1 -- 172.21.15.112:0/2949496259 wait complete. 2024-04-24T23:05:03.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:03 smithi112 ceph-mon[29186]: [24/Apr/2024:23:05:01] ENGINE Bus STARTING 2024-04-24T23:05:03.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:03 smithi112 ceph-mon[29186]: [24/Apr/2024:23:05:01] ENGINE Serving on https://172.21.15.112:7150 2024-04-24T23:05:03.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:03 smithi112 ceph-mon[29186]: [24/Apr/2024:23:05:01] ENGINE Serving on http://172.21.15.112:8765 2024-04-24T23:05:03.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:03 smithi112 ceph-mon[29186]: [24/Apr/2024:23:05:01] ENGINE Bus STARTED 2024-04-24T23:05:03.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:03 smithi112 ceph-mon[29186]: mgrmap e8: x(active, since 2s) 2024-04-24T23:05:03.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:03 smithi112 ceph-mon[29186]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:03.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:03 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:03.472 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:03 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:04.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T23:05:04.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.481+0000 7f65eb2d4700 1 Processor -- start 2024-04-24T23:05:04.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65eb2d4700 1 -- start start 2024-04-24T23:05:04.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65eb2d4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65eb2d4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65e407c870 con 0x7f65e407de40 2024-04-24T23:05:04.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65e9070700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65e9070700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e407c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35942/0 (socket says 172.21.15.112:35942) 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65e9070700 1 -- 172.21.15.112:0/869241939 learned_addr learned my addr 172.21.15.112:0/869241939 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65e9070700 1 -- 172.21.15.112:0/869241939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e407c9f0 con 0x7f65e407de40 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65e9070700 1 --2- 172.21.15.112:0/869241939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e407c2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f65d40098a0 tx=0x7f65d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba49dcd5a5675f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.482+0000 7f65dbfff700 1 -- 172.21.15.112:0/869241939 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d4004160 con 0x7f65e407de40 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65dbfff700 1 -- 172.21.15.112:0/869241939 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f65d4004300 con 0x7f65e407de40 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65dbfff700 1 -- 172.21.15.112:0/869241939 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d4011c00 con 0x7f65e407de40 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65eb2d4700 1 -- 172.21.15.112:0/869241939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 msgr2=0x7f65e407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65eb2d4700 1 --2- 172.21.15.112:0/869241939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e407c2a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f65d40098a0 tx=0x7f65d4009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65eb2d4700 1 -- 172.21.15.112:0/869241939 shutdown_connections 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65eb2d4700 1 --2- 172.21.15.112:0/869241939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e407c2a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65eb2d4700 1 -- 172.21.15.112:0/869241939 >> 172.21.15.112:0/869241939 conn(0x7f65e41032a0 msgr2=0x7f65e41056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:04.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65eb2d4700 1 -- 172.21.15.112:0/869241939 shutdown_connections 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.483+0000 7f65eb2d4700 1 -- 172.21.15.112:0/869241939 wait complete. 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65eb2d4700 1 Processor -- start 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65eb2d4700 1 -- start start 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65eb2d4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65eb2d4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f65e410e310 con 0x7f65e407de40 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65e9070700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65e9070700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e419def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35948/0 (socket says 172.21.15.112:35948) 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65e9070700 1 -- 172.21.15.112:0/3924477407 learned_addr learned my addr 172.21.15.112:0/3924477407 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65e9070700 1 -- 172.21.15.112:0/3924477407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e419e430 con 0x7f65e407de40 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.484+0000 7f65e9070700 1 --2- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e419def0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009fb0 tx=0x7f65d4011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.485+0000 7f65da7fc700 1 -- 172.21.15.112:0/3924477407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d40038b0 con 0x7f65e407de40 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.485+0000 7f65da7fc700 1 -- 172.21.15.112:0/3924477407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f65d401a430 con 0x7f65e407de40 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.485+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e419e6c0 con 0x7f65e407de40 2024-04-24T23:05:04.243 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.485+0000 7f65da7fc700 1 -- 172.21.15.112:0/3924477407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d40195e0 con 0x7f65e407de40 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.485+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e41a13b0 con 0x7f65e407de40 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.485+0000 7f65da7fc700 1 -- 172.21.15.112:0/3924477407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f65d4019890 con 0x7f65e407de40 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.485+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e4063890 con 0x7f65e407de40 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.486+0000 7f65da7fc700 1 --2- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f65d0039050 0x7f65d003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.486+0000 7f65da7fc700 1 -- 172.21.15.112:0/3924477407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f65d404d360 con 0x7f65e407de40 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.486+0000 7f65e886f700 1 --2- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f65d0039050 0x7f65d003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.486+0000 7f65e886f700 1 --2- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f65d0039050 0x7f65d003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006fd0 tx=0x7f65e0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.491+0000 7f65da7fc700 1 -- 172.21.15.112:0/3924477407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f65d4020c80 con 0x7f65e407de40 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.688+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f65e41044e0 con 0x7f65d0039050 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.688+0000 7f65da7fc700 1 -- 172.21.15.112:0/3924477407 <== mgr.14118 v2:172.21.15.112:6800/2319978079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f65e41044e0 con 0x7f65d0039050 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.690+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f65d0039050 msgr2=0x7f65d003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.690+0000 7f65eb2d4700 1 --2- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f65d0039050 0x7f65d003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006fd0 tx=0x7f65e0006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.690+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 msgr2=0x7f65e419def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.690+0000 7f65eb2d4700 1 --2- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e419def0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009fb0 tx=0x7f65d4011c60 comp rx=0 tx=0).stop 2024-04-24T23:05:04.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.690+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 shutdown_connections 2024-04-24T23:05:04.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.690+0000 7f65eb2d4700 1 --2- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f65d0039050 0x7f65d003b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.690+0000 7f65eb2d4700 1 --2- 172.21.15.112:0/3924477407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f65e407de40 0x7f65e419def0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:04.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.690+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 >> 172.21.15.112:0/3924477407 conn(0x7f65e41032a0 msgr2=0x7f65e4103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:04.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.691+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 shutdown_connections 2024-04-24T23:05:04.245 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:03.691+0000 7f65eb2d4700 1 -- 172.21.15.112:0/3924477407 wait complete. 2024-04-24T23:05:04.245 INFO:teuthology.orchestra.run.smithi112.stdout:Generating ssh key... 2024-04-24T23:05:05.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:04 smithi112 ceph-mon[29186]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:05.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: Generating public/private rsa key pair. 2024-04-24T23:05:05.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: Your identification has been saved in /tmp/tmp6a6remtq/key. 2024-04-24T23:05:05.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: Your public key has been saved in /tmp/tmp6a6remtq/key.pub. 2024-04-24T23:05:05.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: The key fingerprint is: 2024-04-24T23:05:05.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: SHA256:vcfy/qxRo8+zfsVsIYgfLLjSetXL/TQ62QCxOjSeGc8 ceph-e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: The key's randomart image is: 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: +---[RSA 3072]----+ 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | | 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | . | 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | . o + | 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | . B * . . | 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | . S & o +o.| 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | . o B E + .=| 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | o . = B ++.| 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | . . * O+.o| 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: | . .+=X= | 2024-04-24T23:05:05.530 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: +----[SHA256]-----+ 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.661+0000 7f894c862700 1 Processor -- start 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.661+0000 7f894c862700 1 -- start start 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.661+0000 7f894c862700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894810ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894c862700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f894807bdf0 con 0x7f8948108780 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894810ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894810ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35962/0 (socket says 172.21.15.112:35962) 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894659c700 1 -- 172.21.15.112:0/2268051655 learned_addr learned my addr 172.21.15.112:0/2268051655 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894659c700 1 -- 172.21.15.112:0/2268051655 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894810b0b0 con 0x7f8948108780 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894659c700 1 --2- 172.21.15.112:0/2268051655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894810ab70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f89380098a0 tx=0x7f8938009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a532f503d41e5427 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:05.730 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894559a700 1 -- 172.21.15.112:0/2268051655 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8938004160 con 0x7f8948108780 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894559a700 1 -- 172.21.15.112:0/2268051655 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f8938004300 con 0x7f8948108780 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.662+0000 7f894559a700 1 -- 172.21.15.112:0/2268051655 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8938011c00 con 0x7f8948108780 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.663+0000 7f894c862700 1 -- 172.21.15.112:0/2268051655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 msgr2=0x7f894810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.663+0000 7f894c862700 1 --2- 172.21.15.112:0/2268051655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894810ab70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f89380098a0 tx=0x7f8938009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.663+0000 7f894c862700 1 -- 172.21.15.112:0/2268051655 shutdown_connections 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.663+0000 7f894c862700 1 --2- 172.21.15.112:0/2268051655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894810ab70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.663+0000 7f894c862700 1 -- 172.21.15.112:0/2268051655 >> 172.21.15.112:0/2268051655 conn(0x7f8948102190 msgr2=0x7f89481045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.663+0000 7f894c862700 1 -- 172.21.15.112:0/2268051655 shutdown_connections 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.663+0000 7f894c862700 1 -- 172.21.15.112:0/2268051655 wait complete. 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894c862700 1 Processor -- start 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894c862700 1 -- start start 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894c862700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894c862700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f894810c130 con 0x7f8948108780 2024-04-24T23:05:05.731 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894819bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35976/0 (socket says 172.21.15.112:35976) 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894659c700 1 -- 172.21.15.112:0/2608711476 learned_addr learned my addr 172.21.15.112:0/2608711476 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894659c700 1 -- 172.21.15.112:0/2608711476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894819c220 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894659c700 1 --2- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894819bce0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8938009fb0 tx=0x7f8938011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f89377fe700 1 -- 172.21.15.112:0/2608711476 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89380038b0 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f89377fe700 1 -- 172.21.15.112:0/2608711476 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f893801a430 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894819c4b0 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.664+0000 7f89377fe700 1 -- 172.21.15.112:0/2608711476 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8938019630 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.665+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894819f190 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.665+0000 7f89377fe700 1 -- 172.21.15.112:0/2608711476 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8938019930 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.665+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8948063890 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.665+0000 7f89377fe700 1 --2- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f8930039000 0x7f893003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.666+0000 7f89377fe700 1 -- 172.21.15.112:0/2608711476 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f893804d3a0 con 0x7f8948108780 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.666+0000 7f8945d9b700 1 --2- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f8930039000 0x7f893003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:05.732 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.666+0000 7f8945d9b700 1 --2- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f8930039000 0x7f893003b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f893c006fd0 tx=0x7f893c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.671+0000 7f89377fe700 1 -- 172.21.15.112:0/2608711476 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f893800b9c0 con 0x7f8948108780 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:04.865+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8948103390 con 0x7f8930039000 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.156+0000 7f89377fe700 1 -- 172.21.15.112:0/2608711476 <== mgr.14118 v2:172.21.15.112:6800/2319978079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8948103390 con 0x7f8930039000 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f8930039000 msgr2=0x7f893003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 --2- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f8930039000 0x7f893003b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f893c006fd0 tx=0x7f893c006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 msgr2=0x7f894819bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 --2- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894819bce0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8938009fb0 tx=0x7f8938011c60 comp rx=0 tx=0).stop 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 shutdown_connections 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 --2- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f8930039000 0x7f893003b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 --2- 172.21.15.112:0/2608711476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8948108780 0x7f894819bce0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 >> 172.21.15.112:0/2608711476 conn(0x7f8948102190 msgr2=0x7f8948102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 shutdown_connections 2024-04-24T23:05:05.733 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:05.158+0000 7f894c862700 1 -- 172.21.15.112:0/2608711476 wait complete. 2024-04-24T23:05:06.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:06 smithi112 ceph-mon[29186]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:06.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:06 smithi112 ceph-mon[29186]: Generating ssh key... 2024-04-24T23:05:06.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:06 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:06.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:06 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:06.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNsvObsMR//qQxiYg0j+66E0veSpCPpxanjsMSWnVruZe2n5pwrfgoST0EvgNBaluGK1wuF+GoebwYhuBVG1/6WbZImCU7ROJvjkFXi8+fqO6IRybtlh1/fQfLz6wsnuIdkZxcY6mFyi/KrbXgHbs7whi0R3hEnqdrIaZH8aFBYDApSGzk5+q/Nk1a4yBxRaL3ZXmecdRH0uwQdn0XqD2n9xLuTum1CJCkrR0+7QgDM6NAiL6x5gdrg1ORahqSspuilOzSncKKFQso1VZid+jP0RvpFjLXafEiNcxWE5GX0LKE9i1tuOS4fDoE6S39iWJmm7JczKWfveBEZi7D8GkJ2XJajgOaVTSjqaGpAHh+7+0KqedaO0IhCs0yvTsg5Wds2uTyvDIAe+VGUbjOMi9yS+3gPgYDUzvT3kALseFRuJUFqYgPmiTVwxAKp+hLJ9DtXWhBWdpQN50rsfu72Pp9ApBIomDxeff4SIFkkoMZsaw7KjselH7dnRunFJ9e0Rc= ceph-e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:05:06.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.171+0000 7f1b89668700 1 Processor -- start 2024-04-24T23:05:06.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b89668700 1 -- start start 2024-04-24T23:05:06.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b89668700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b84106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b89668700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b841075a0 con 0x7f1b84106bb0 2024-04-24T23:05:06.995 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b82ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b84106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b82ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b84106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35992/0 (socket says 172.21.15.112:35992) 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b82ffd700 1 -- 172.21.15.112:0/3386290756 learned_addr learned my addr 172.21.15.112:0/3386290756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b82ffd700 1 -- 172.21.15.112:0/3386290756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b84107db0 con 0x7f1b84106bb0 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b82ffd700 1 --2- 172.21.15.112:0/3386290756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b84106fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b740098a0 tx=0x7f1b74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9288a9ce4a6b36e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.172+0000 7f1b81ffb700 1 -- 172.21.15.112:0/3386290756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b74004160 con 0x7f1b84106bb0 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b81ffb700 1 -- 172.21.15.112:0/3386290756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1b74004300 con 0x7f1b84106bb0 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b81ffb700 1 -- 172.21.15.112:0/3386290756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b74011c00 con 0x7f1b84106bb0 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b89668700 1 -- 172.21.15.112:0/3386290756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 msgr2=0x7f1b84106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b89668700 1 --2- 172.21.15.112:0/3386290756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b84106fd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b740098a0 tx=0x7f1b74009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b89668700 1 -- 172.21.15.112:0/3386290756 shutdown_connections 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b89668700 1 --2- 172.21.15.112:0/3386290756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b84106fd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b89668700 1 -- 172.21.15.112:0/3386290756 >> 172.21.15.112:0/3386290756 conn(0x7f1b84102170 msgr2=0x7f1b84104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:06.996 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b89668700 1 -- 172.21.15.112:0/3386290756 shutdown_connections 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.173+0000 7f1b89668700 1 -- 172.21.15.112:0/3386290756 wait complete. 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.174+0000 7f1b89668700 1 Processor -- start 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.174+0000 7f1b89668700 1 -- start start 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.174+0000 7f1b89668700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b8419bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.174+0000 7f1b89668700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8410b160 con 0x7f1b84106bb0 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.174+0000 7f1b82ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b8419bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.174+0000 7f1b82ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b8419bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36002/0 (socket says 172.21.15.112:36002) 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.174+0000 7f1b82ffd700 1 -- 172.21.15.112:0/2528424546 learned_addr learned my addr 172.21.15.112:0/2528424546 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.175+0000 7f1b82ffd700 1 -- 172.21.15.112:0/2528424546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8419c330 con 0x7f1b84106bb0 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.175+0000 7f1b82ffd700 1 --2- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b8419bdf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009fb0 tx=0x7f1b74011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.175+0000 7f1b6bfff700 1 -- 172.21.15.112:0/2528424546 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b740038b0 con 0x7f1b84106bb0 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.175+0000 7f1b6bfff700 1 -- 172.21.15.112:0/2528424546 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f1b7401a430 con 0x7f1b84106bb0 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.175+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8419c5c0 con 0x7f1b84106bb0 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.175+0000 7f1b6bfff700 1 -- 172.21.15.112:0/2528424546 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b740195e0 con 0x7f1b84106bb0 2024-04-24T23:05:06.997 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.175+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8419f220 con 0x7f1b84106bb0 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.176+0000 7f1b6bfff700 1 -- 172.21.15.112:0/2528424546 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1b740198e0 con 0x7f1b84106bb0 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.176+0000 7f1b6bfff700 1 --2- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f1b6c039000 0x7f1b6c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.176+0000 7f1b6bfff700 1 -- 172.21.15.112:0/2528424546 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1b7404e500 con 0x7f1b84106bb0 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.176+0000 7f1b827fc700 1 --2- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f1b6c039000 0x7f1b6c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.176+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8410b2e0 con 0x7f1b84106bb0 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.176+0000 7f1b827fc700 1 --2- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f1b6c039000 0x7f1b6c03b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b78006fd0 tx=0x7f1b78006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.181+0000 7f1b6bfff700 1 -- 172.21.15.112:0/2528424546 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1b7400fda0 con 0x7f1b84106bb0 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.375+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f1b8402de10 con 0x7f1b6c039000 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.376+0000 7f1b6bfff700 1 -- 172.21.15.112:0/2528424546 <== mgr.14118 v2:172.21.15.112:6800/2319978079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f1b8402de10 con 0x7f1b6c039000 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.378+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f1b6c039000 msgr2=0x7f1b6c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.378+0000 7f1b89668700 1 --2- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f1b6c039000 0x7f1b6c03b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1b78006fd0 tx=0x7f1b78006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.378+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 msgr2=0x7f1b8419bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.378+0000 7f1b89668700 1 --2- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b8419bdf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009fb0 tx=0x7f1b74011c60 comp rx=0 tx=0).stop 2024-04-24T23:05:06.998 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.378+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 shutdown_connections 2024-04-24T23:05:06.999 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.378+0000 7f1b89668700 1 --2- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f1b6c039000 0x7f1b6c03b4c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.999 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.378+0000 7f1b89668700 1 --2- 172.21.15.112:0/2528424546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b84106bb0 0x7f1b8419bdf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.999 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.378+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 >> 172.21.15.112:0/2528424546 conn(0x7f1b84102170 msgr2=0x7f1b841030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:06.999 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.379+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 shutdown_connections 2024-04-24T23:05:06.999 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:06.379+0000 7f1b89668700 1 -- 172.21.15.112:0/2528424546 wait complete. 2024-04-24T23:05:06.999 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:05:06.999 INFO:teuthology.orchestra.run.smithi112.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T23:05:06.999 INFO:teuthology.orchestra.run.smithi112.stdout:Adding host smithi112... 2024-04-24T23:05:07.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:07 smithi112 ceph-mon[29186]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:09.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:08 smithi112 ceph-mon[29186]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:09.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:08 smithi112 ceph-mon[29186]: Deploying cephadm binary to smithi112 2024-04-24T23:05:12.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Added host 'smithi112' with addr '172.21.15.112' 2024-04-24T23:05:12.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.471+0000 7f50a0d6d700 1 Processor -- start 2024-04-24T23:05:12.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.471+0000 7f50a0d6d700 1 -- start start 2024-04-24T23:05:12.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.471+0000 7f50a0d6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c1074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.471+0000 7f50a0d6d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f509c107a90 con 0x7f509c1070a0 2024-04-24T23:05:12.278 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.471+0000 7f509a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c1074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.471+0000 7f509a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c1074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36004/0 (socket says 172.21.15.112:36004) 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.471+0000 7f509a59c700 1 -- 172.21.15.112:0/972849127 learned_addr learned my addr 172.21.15.112:0/972849127 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.471+0000 7f509a59c700 1 -- 172.21.15.112:0/972849127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c107dc0 con 0x7f509c1070a0 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.472+0000 7f509a59c700 1 --2- 172.21.15.112:0/972849127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c1074c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f508c00bb40 tx=0x7f508c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e701946c9575675 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.472+0000 7f509959a700 1 -- 172.21.15.112:0/972849127 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f508c00dbe0 con 0x7f509c1070a0 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.472+0000 7f509959a700 1 -- 172.21.15.112:0/972849127 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f508c004510 con 0x7f509c1070a0 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.472+0000 7f509959a700 1 -- 172.21.15.112:0/972849127 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f508c0104d0 con 0x7f509c1070a0 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.472+0000 7f50a0d6d700 1 -- 172.21.15.112:0/972849127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 msgr2=0x7f509c1074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.472+0000 7f50a0d6d700 1 --2- 172.21.15.112:0/972849127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c1074c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f508c00bb40 tx=0x7f508c00be50 comp rx=0 tx=0).stop 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 -- 172.21.15.112:0/972849127 shutdown_connections 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 --2- 172.21.15.112:0/972849127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c1074c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 -- 172.21.15.112:0/972849127 >> 172.21.15.112:0/972849127 conn(0x7f509c101900 msgr2=0x7f509c103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 -- 172.21.15.112:0/972849127 shutdown_connections 2024-04-24T23:05:12.279 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 -- 172.21.15.112:0/972849127 wait complete. 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 Processor -- start 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 -- start start 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.473+0000 7f50a0d6d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f509c105d80 con 0x7f509c1070a0 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f509a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f509a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36006/0 (socket says 172.21.15.112:36006) 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f509a59c700 1 -- 172.21.15.112:0/2643815687 learned_addr learned my addr 172.21.15.112:0/2643815687 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f509a59c700 1 -- 172.21.15.112:0/2643815687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c199fc0 con 0x7f509c1070a0 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f509a59c700 1 --2- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c199a80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f508c0046e0 tx=0x7f508c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f508b7fe700 1 -- 172.21.15.112:0/2643815687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f508c027070 con 0x7f509c1070a0 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f508b7fe700 1 -- 172.21.15.112:0/2643815687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7f508c004210 con 0x7f509c1070a0 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f508b7fe700 1 -- 172.21.15.112:0/2643815687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f508c022b70 con 0x7f509c1070a0 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509c19a250 con 0x7f509c1070a0 2024-04-24T23:05:12.280 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.474+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509c19cef0 con 0x7f509c1070a0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.475+0000 7f508b7fe700 1 -- 172.21.15.112:0/2643815687 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f508c02a070 con 0x7f509c1070a0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.475+0000 7f508b7fe700 1 --2- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f50840390a0 0x7f508403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.475+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509c063890 con 0x7f509c1070a0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.475+0000 7f508b7fe700 1 -- 172.21.15.112:0/2643815687 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f508c01f070 con 0x7f509c1070a0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.475+0000 7f5099d9b700 1 --2- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f50840390a0 0x7f508403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.476+0000 7f5099d9b700 1 --2- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f50840390a0 0x7f508403b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5090006fd0 tx=0x7f5090006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.480+0000 7f508b7fe700 1 -- 172.21.15.112:0/2643815687 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f508c022e20 con 0x7f509c1070a0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:07.679+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}) -- 0x7f509c062670 con 0x7f50840390a0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.740+0000 7f508b7fe700 1 -- 172.21.15.112:0/2643815687 <== mgr.14118 v2:172.21.15.112:6800/2319978079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f509c062670 con 0x7f50840390a0 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f50840390a0 msgr2=0x7f508403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 --2- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f50840390a0 0x7f508403b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5090006fd0 tx=0x7f5090006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:12.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 msgr2=0x7f509c199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 --2- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c199a80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f508c0046e0 tx=0x7f508c00dd70 comp rx=0 tx=0).stop 2024-04-24T23:05:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 shutdown_connections 2024-04-24T23:05:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 --2- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f50840390a0 0x7f508403b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 --2- 172.21.15.112:0/2643815687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f509c1070a0 0x7f509c199a80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 >> 172.21.15.112:0/2643815687 conn(0x7f509c101900 msgr2=0x7f509c1900a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 shutdown_connections 2024-04-24T23:05:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:11.743+0000 7f50a0d6d700 1 -- 172.21.15.112:0/2643815687 wait complete. 2024-04-24T23:05:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mon service... 2024-04-24T23:05:12.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:12 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:12.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:12 smithi112 ceph-mon[29186]: Added host smithi112 2024-04-24T23:05:12.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:12 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:13.829 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T23:05:13.829 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.713+0000 7fcc1bef8700 1 Processor -- start 2024-04-24T23:05:13.829 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.713+0000 7fcc1bef8700 1 -- start start 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.713+0000 7fcc1bef8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1410aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.713+0000 7fcc1bef8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1407bdd0 con 0x7fcc141087b0 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.713+0000 7fcc19c94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1410aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.714+0000 7fcc19c94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1410aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33530/0 (socket says 172.21.15.112:33530) 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.714+0000 7fcc19c94700 1 -- 172.21.15.112:0/3654886442 learned_addr learned my addr 172.21.15.112:0/3654886442 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.714+0000 7fcc19c94700 1 -- 172.21.15.112:0/3654886442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1410b0e0 con 0x7fcc141087b0 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.714+0000 7fcc19c94700 1 --2- 172.21.15.112:0/3654886442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1410aba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098a0 tx=0x7fcc10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10edb8690a155c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.714+0000 7fcc18c92700 1 -- 172.21.15.112:0/3654886442 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10004160 con 0x7fcc141087b0 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.714+0000 7fcc18c92700 1 -- 172.21.15.112:0/3654886442 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fcc10004300 con 0x7fcc141087b0 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.714+0000 7fcc18c92700 1 -- 172.21.15.112:0/3654886442 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10011c00 con 0x7fcc141087b0 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 -- 172.21.15.112:0/3654886442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 msgr2=0x7fcc1410aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 --2- 172.21.15.112:0/3654886442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1410aba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098a0 tx=0x7fcc10009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.830 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 -- 172.21.15.112:0/3654886442 shutdown_connections 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 --2- 172.21.15.112:0/3654886442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1410aba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 -- 172.21.15.112:0/3654886442 >> 172.21.15.112:0/3654886442 conn(0x7fcc141021a0 msgr2=0x7fcc141045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 -- 172.21.15.112:0/3654886442 shutdown_connections 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 -- 172.21.15.112:0/3654886442 wait complete. 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 Processor -- start 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.715+0000 7fcc1bef8700 1 -- start start 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc1bef8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc1bef8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1410c110 con 0x7fcc141087b0 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc19c94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc19c94700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33536/0 (socket says 172.21.15.112:33536) 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc19c94700 1 -- 172.21.15.112:0/2228262110 learned_addr learned my addr 172.21.15.112:0/2228262110 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc19c94700 1 -- 172.21.15.112:0/2228262110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1419c250 con 0x7fcc141087b0 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc19c94700 1 --2- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1419bd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009fb0 tx=0x7fcc10011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc0affd700 1 -- 172.21.15.112:0/2228262110 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc100038b0 con 0x7fcc141087b0 2024-04-24T23:05:13.831 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc0affd700 1 -- 172.21.15.112:0/2228262110 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7fcc1001a430 con 0x7fcc141087b0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1419c4e0 con 0x7fcc141087b0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc0affd700 1 -- 172.21.15.112:0/2228262110 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10019630 con 0x7fcc141087b0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.716+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1419f1c0 con 0x7fcc141087b0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.717+0000 7fcc0affd700 1 -- 172.21.15.112:0/2228262110 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcc10019930 con 0x7fcc141087b0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.717+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc14063890 con 0x7fcc141087b0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.717+0000 7fcc0affd700 1 --2- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fcc00039000 0x7fcc0003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.717+0000 7fcc0affd700 1 -- 172.21.15.112:0/2228262110 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcc1004d4d0 con 0x7fcc141087b0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.717+0000 7fcc19493700 1 --2- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fcc00039000 0x7fcc0003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.718+0000 7fcc19493700 1 --2- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fcc00039000 0x7fcc0003b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc04006fd0 tx=0x7fcc04006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.722+0000 7fcc0affd700 1 -- 172.21.15.112:0/2228262110 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc10019d20 con 0x7fcc141087b0 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.934+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcc14103410 con 0x7fcc00039000 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.939+0000 7fcc0affd700 1 -- 172.21.15.112:0/2228262110 <== mgr.14118 v2:172.21.15.112:6800/2319978079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcc14103410 con 0x7fcc00039000 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fcc00039000 msgr2=0x7fcc0003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.832 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 --2- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fcc00039000 0x7fcc0003b4c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcc04006fd0 tx=0x7fcc04006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 msgr2=0x7fcc1419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 --2- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1419bd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009fb0 tx=0x7fcc10011c60 comp rx=0 tx=0).stop 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 shutdown_connections 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 --2- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fcc00039000 0x7fcc0003b4c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 --2- 172.21.15.112:0/2228262110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc141087b0 0x7fcc1419bd10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 >> 172.21.15.112:0/2228262110 conn(0x7fcc141021a0 msgr2=0x7fcc14102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 shutdown_connections 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:12.941+0000 7fcc1bef8700 1 -- 172.21.15.112:0/2228262110 wait complete. 2024-04-24T23:05:13.833 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mgr service... 2024-04-24T23:05:14.149 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:13 smithi112 ceph-mon[29186]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:14.149 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:13 smithi112 ceph-mon[29186]: Saving service mon spec with placement count:5 2024-04-24T23:05:14.149 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:13 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:15.648 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:15 smithi112 ceph-mon[29186]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:15.648 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:15 smithi112 ceph-mon[29186]: Saving service mgr spec with placement count:2 2024-04-24T23:05:15.648 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:15 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:15.648 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:15 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:15.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T23:05:15.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.249+0000 7ff8f4f79700 1 Processor -- start 2024-04-24T23:05:15.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.249+0000 7ff8f4f79700 1 -- start start 2024-04-24T23:05:15.656 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.249+0000 7ff8f4f79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f0106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.249+0000 7ff8f4f79700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01075a0 con 0x7ff8f0106bb0 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.249+0000 7ff8ee59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f0106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.249+0000 7ff8ee59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f0106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33544/0 (socket says 172.21.15.112:33544) 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.249+0000 7ff8ee59c700 1 -- 172.21.15.112:0/3513895230 learned_addr learned my addr 172.21.15.112:0/3513895230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.249+0000 7ff8ee59c700 1 -- 172.21.15.112:0/3513895230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f0107db0 con 0x7ff8f0106bb0 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8ee59c700 1 --2- 172.21.15.112:0/3513895230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f0106fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00098a0 tx=0x7ff8e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7487a16896cee17b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8ed59a700 1 -- 172.21.15.112:0/3513895230 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e0004160 con 0x7ff8f0106bb0 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8ed59a700 1 -- 172.21.15.112:0/3513895230 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ff8e0004300 con 0x7ff8f0106bb0 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8ed59a700 1 -- 172.21.15.112:0/3513895230 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e0011c00 con 0x7ff8f0106bb0 2024-04-24T23:05:15.657 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8f4f79700 1 -- 172.21.15.112:0/3513895230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 msgr2=0x7ff8f0106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8f4f79700 1 --2- 172.21.15.112:0/3513895230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f0106fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00098a0 tx=0x7ff8e0009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8f4f79700 1 -- 172.21.15.112:0/3513895230 shutdown_connections 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8f4f79700 1 --2- 172.21.15.112:0/3513895230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f0106fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8f4f79700 1 -- 172.21.15.112:0/3513895230 >> 172.21.15.112:0/3513895230 conn(0x7ff8f0102170 msgr2=0x7ff8f0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8f4f79700 1 -- 172.21.15.112:0/3513895230 shutdown_connections 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.250+0000 7ff8f4f79700 1 -- 172.21.15.112:0/3513895230 wait complete. 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.251+0000 7ff8f4f79700 1 Processor -- start 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.251+0000 7ff8f4f79700 1 -- start start 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.251+0000 7ff8f4f79700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.251+0000 7ff8f4f79700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f010b160 con 0x7ff8f0106bb0 2024-04-24T23:05:15.658 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.251+0000 7ff8ee59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.251+0000 7ff8ee59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33546/0 (socket says 172.21.15.112:33546) 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.251+0000 7ff8ee59c700 1 -- 172.21.15.112:0/1863344004 learned_addr learned my addr 172.21.15.112:0/1863344004 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.252+0000 7ff8ee59c700 1 -- 172.21.15.112:0/1863344004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f019b1a0 con 0x7ff8f0106bb0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.252+0000 7ff8ee59c700 1 --2- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f019ac60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009fb0 tx=0x7ff8e0011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.252+0000 7ff8df7fe700 1 -- 172.21.15.112:0/1863344004 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e00038b0 con 0x7ff8f0106bb0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.252+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f019b430 con 0x7ff8f0106bb0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.252+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f019e120 con 0x7ff8f0106bb0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.252+0000 7ff8df7fe700 1 -- 172.21.15.112:0/1863344004 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1143+0+0 (secure 0 0 0) 0x7ff8e001a430 con 0x7ff8f0106bb0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.252+0000 7ff8df7fe700 1 -- 172.21.15.112:0/1863344004 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8e00195e0 con 0x7ff8f0106bb0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.253+0000 7ff8df7fe700 1 -- 172.21.15.112:0/1863344004 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff8e0019910 con 0x7ff8f0106bb0 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.253+0000 7ff8df7fe700 1 --2- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff8d8039050 0x7ff8d803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:15.659 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.253+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f010b2e0 con 0x7ff8f0106bb0 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.253+0000 7ff8df7fe700 1 -- 172.21.15.112:0/1863344004 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff8e004d3a0 con 0x7ff8f0106bb0 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.253+0000 7ff8edd9b700 1 --2- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff8d8039050 0x7ff8d803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.253+0000 7ff8edd9b700 1 --2- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff8d8039050 0x7ff8d803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4006fd0 tx=0x7ff8e4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.258+0000 7ff8df7fe700 1 -- 172.21.15.112:0/1863344004 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8e0016dd0 con 0x7ff8f0106bb0 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.472+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff8f0103480 con 0x7ff8d8039050 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.477+0000 7ff8df7fe700 1 -- 172.21.15.112:0/1863344004 <== mgr.14118 v2:172.21.15.112:6800/2319978079 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff8f0103480 con 0x7ff8d8039050 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff8d8039050 msgr2=0x7ff8d803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 --2- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff8d8039050 0x7ff8d803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4006fd0 tx=0x7ff8e4006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 msgr2=0x7ff8f019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:15.660 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 --2- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f019ac60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009fb0 tx=0x7ff8e0011c60 comp rx=0 tx=0).stop 2024-04-24T23:05:15.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 shutdown_connections 2024-04-24T23:05:15.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 --2- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff8d8039050 0x7ff8d803b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 --2- 172.21.15.112:0/1863344004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8f0106bb0 0x7ff8f019ac60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:15.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 >> 172.21.15.112:0/1863344004 conn(0x7ff8f0102170 msgr2=0x7ff8f0102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:15.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 shutdown_connections 2024-04-24T23:05:15.661 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:14.479+0000 7ff8f4f79700 1 -- 172.21.15.112:0/1863344004 wait complete. 2024-04-24T23:05:16.902 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:16 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:16.902 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:16 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/289065029' entity='client.admin' 2024-04-24T23:05:17.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.103+0000 7f6e26f34700 1 Processor -- start 2024-04-24T23:05:17.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.103+0000 7f6e26f34700 1 -- start start 2024-04-24T23:05:17.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.103+0000 7f6e26f34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2010ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.103+0000 7f6e26f34700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2007be30 con 0x7f6e20108740 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.103+0000 7f6e24cd0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2010ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.103+0000 7f6e24cd0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2010ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33552/0 (socket says 172.21.15.112:33552) 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.103+0000 7f6e24cd0700 1 -- 172.21.15.112:0/555908805 learned_addr learned my addr 172.21.15.112:0/555908805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.104+0000 7f6e24cd0700 1 -- 172.21.15.112:0/555908805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2010b070 con 0x7f6e20108740 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.104+0000 7f6e24cd0700 1 --2- 172.21.15.112:0/555908805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2010ab30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e140098a0 tx=0x7f6e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f116dd64666528 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.104+0000 7f6e1f7fe700 1 -- 172.21.15.112:0/555908805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e14004160 con 0x7f6e20108740 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.104+0000 7f6e1f7fe700 1 -- 172.21.15.112:0/555908805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6e14004300 con 0x7f6e20108740 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.104+0000 7f6e1f7fe700 1 -- 172.21.15.112:0/555908805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e14011bb0 con 0x7f6e20108740 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 -- 172.21.15.112:0/555908805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 msgr2=0x7f6e2010ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 --2- 172.21.15.112:0/555908805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2010ab30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e140098a0 tx=0x7f6e14009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 -- 172.21.15.112:0/555908805 shutdown_connections 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 --2- 172.21.15.112:0/555908805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2010ab30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 -- 172.21.15.112:0/555908805 >> 172.21.15.112:0/555908805 conn(0x7f6e20102190 msgr2=0x7f6e201045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:17.239 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 -- 172.21.15.112:0/555908805 shutdown_connections 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 -- 172.21.15.112:0/555908805 wait complete. 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 Processor -- start 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.105+0000 7f6e26f34700 1 -- start start 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e26f34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e26f34700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2010c140 con 0x7f6e20108740 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e24cd0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e24cd0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2019bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33564/0 (socket says 172.21.15.112:33564) 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e24cd0700 1 -- 172.21.15.112:0/289065029 learned_addr learned my addr 172.21.15.112:0/289065029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e24cd0700 1 -- 172.21.15.112:0/289065029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2019c1e0 con 0x7f6e20108740 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e24cd0700 1 --2- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2019bca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6e14000c00 tx=0x7f6e14011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e1dffb700 1 -- 172.21.15.112:0/289065029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e14011e60 con 0x7f6e20108740 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e1dffb700 1 -- 172.21.15.112:0/289065029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6e14020070 con 0x7f6e20108740 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e1dffb700 1 -- 172.21.15.112:0/289065029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e14019550 con 0x7f6e20108740 2024-04-24T23:05:17.240 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2019c470 con 0x7f6e20108740 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.106+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2019c8d0 con 0x7f6e20108740 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.107+0000 7f6e1dffb700 1 -- 172.21.15.112:0/289065029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6e14003aa0 con 0x7f6e20108740 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.107+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e20063890 con 0x7f6e20108740 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.107+0000 7f6e1dffb700 1 --2- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f6e100390a0 0x7f6e1003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.108+0000 7f6e1dffb700 1 -- 172.21.15.112:0/289065029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6e1404d2e0 con 0x7f6e20108740 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.108+0000 7f6e1ffff700 1 --2- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f6e100390a0 0x7f6e1003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.108+0000 7f6e1ffff700 1 --2- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f6e100390a0 0x7f6e1003b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006fd0 tx=0x7f6e0c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.113+0000 7f6e1dffb700 1 -- 172.21.15.112:0/289065029 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6e1400e260 con 0x7f6e20108740 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.326+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6e2019f710 con 0x7f6e20108740 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.329+0000 7f6e1dffb700 1 -- 172.21.15.112:0/289065029 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6e1401a950 con 0x7f6e20108740 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f6e100390a0 msgr2=0x7f6e1003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 --2- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f6e100390a0 0x7f6e1003b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c006fd0 tx=0x7f6e0c006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 msgr2=0x7f6e2019bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.241 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 --2- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2019bca0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6e14000c00 tx=0x7f6e14011c30 comp rx=0 tx=0).stop 2024-04-24T23:05:17.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 shutdown_connections 2024-04-24T23:05:17.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 --2- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7f6e100390a0 0x7f6e1003b560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 --2- 172.21.15.112:0/289065029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e20108740 0x7f6e2019bca0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 >> 172.21.15.112:0/289065029 conn(0x7f6e20102190 msgr2=0x7f6e20102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:17.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 shutdown_connections 2024-04-24T23:05:17.242 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:16.332+0000 7f6e26f34700 1 -- 172.21.15.112:0/289065029 wait complete. 2024-04-24T23:05:18.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:18 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:18.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:18 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3052441225' entity='client.admin' 2024-04-24T23:05:18.858 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.080+0000 7fae6cf51700 1 Processor -- start 2024-04-24T23:05:18.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.081+0000 7fae6cf51700 1 -- start start 2024-04-24T23:05:18.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.081+0000 7fae6cf51700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae681074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.081+0000 7fae6cf51700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae68107a90 con 0x7fae681070a0 2024-04-24T23:05:18.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.081+0000 7fae6659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae681074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.859 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.081+0000 7fae6659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae681074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33576/0 (socket says 172.21.15.112:33576) 2024-04-24T23:05:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.081+0000 7fae6659c700 1 -- 172.21.15.112:0/3258127240 learned_addr learned my addr 172.21.15.112:0/3258127240 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.081+0000 7fae6659c700 1 -- 172.21.15.112:0/3258127240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae68107dc0 con 0x7fae681070a0 2024-04-24T23:05:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6659c700 1 --2- 172.21.15.112:0/3258127240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae681074c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae580098a0 tx=0x7fae58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a7f598a93d64de server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6559a700 1 -- 172.21.15.112:0/3258127240 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae58004160 con 0x7fae681070a0 2024-04-24T23:05:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6559a700 1 -- 172.21.15.112:0/3258127240 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae58004300 con 0x7fae681070a0 2024-04-24T23:05:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6559a700 1 -- 172.21.15.112:0/3258127240 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae58011bb0 con 0x7fae681070a0 2024-04-24T23:05:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6cf51700 1 -- 172.21.15.112:0/3258127240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 msgr2=0x7fae681074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6cf51700 1 --2- 172.21.15.112:0/3258127240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae681074c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fae580098a0 tx=0x7fae58009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6cf51700 1 -- 172.21.15.112:0/3258127240 shutdown_connections 2024-04-24T23:05:18.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6cf51700 1 --2- 172.21.15.112:0/3258127240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae681074c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.082+0000 7fae6cf51700 1 -- 172.21.15.112:0/3258127240 >> 172.21.15.112:0/3258127240 conn(0x7fae68101900 msgr2=0x7fae68103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:18.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.083+0000 7fae6cf51700 1 -- 172.21.15.112:0/3258127240 shutdown_connections 2024-04-24T23:05:18.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.083+0000 7fae6cf51700 1 -- 172.21.15.112:0/3258127240 wait complete. 2024-04-24T23:05:18.861 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.083+0000 7fae6cf51700 1 Processor -- start 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.083+0000 7fae6cf51700 1 -- start start 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.083+0000 7fae6cf51700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae6819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.083+0000 7fae6cf51700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae6810b170 con 0x7fae681070a0 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae6659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae6819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae6659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae6819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33580/0 (socket says 172.21.15.112:33580) 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae6659c700 1 -- 172.21.15.112:0/3052441225 learned_addr learned my addr 172.21.15.112:0/3052441225 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae6659c700 1 -- 172.21.15.112:0/3052441225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6819c1c0 con 0x7fae681070a0 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae6659c700 1 --2- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae6819bc80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fae58011870 tx=0x7fae58011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae577fe700 1 -- 172.21.15.112:0/3052441225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae580039f0 con 0x7fae681070a0 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae577fe700 1 -- 172.21.15.112:0/3052441225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae58003b90 con 0x7fae681070a0 2024-04-24T23:05:18.862 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6819c450 con 0x7fae681070a0 2024-04-24T23:05:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae577fe700 1 -- 172.21.15.112:0/3052441225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae5801a800 con 0x7fae681070a0 2024-04-24T23:05:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.084+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6819c8b0 con 0x7fae681070a0 2024-04-24T23:05:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.085+0000 7fae577fe700 1 -- 172.21.15.112:0/3052441225 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fae5801aa00 con 0x7fae681070a0 2024-04-24T23:05:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.085+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae68063890 con 0x7fae681070a0 2024-04-24T23:05:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.085+0000 7fae577fe700 1 --2- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fae50039050 0x7fae5003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.085+0000 7fae577fe700 1 -- 172.21.15.112:0/3052441225 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fae5804d580 con 0x7fae681070a0 2024-04-24T23:05:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.085+0000 7fae65d9b700 1 --2- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fae50039050 0x7fae5003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.086+0000 7fae65d9b700 1 --2- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fae50039050 0x7fae5003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fae5c006fd0 tx=0x7fae5c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.090+0000 7fae577fe700 1 -- 172.21.15.112:0/3052441225 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae58020240 con 0x7fae681070a0 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.303+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fae68062e40 con 0x7fae681070a0 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.305+0000 7fae577fe700 1 -- 172.21.15.112:0/3052441225 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fae5804cf10 con 0x7fae681070a0 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fae50039050 msgr2=0x7fae5003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 --2- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fae50039050 0x7fae5003b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fae5c006fd0 tx=0x7fae5c006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 msgr2=0x7fae6819bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 --2- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae6819bc80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fae58011870 tx=0x7fae58011c30 comp rx=0 tx=0).stop 2024-04-24T23:05:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 shutdown_connections 2024-04-24T23:05:18.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 --2- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fae50039050 0x7fae5003b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 --2- 172.21.15.112:0/3052441225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae681070a0 0x7fae6819bc80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 >> 172.21.15.112:0/3052441225 conn(0x7fae68101900 msgr2=0x7fae681026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:18.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 shutdown_connections 2024-04-24T23:05:18.865 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:18.309+0000 7fae6cf51700 1 -- 172.21.15.112:0/3052441225 wait complete. 2024-04-24T23:05:18.865 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling the dashboard module... 2024-04-24T23:05:20.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:19 smithi112 ceph-mon[29186]: from='mgr.14118 172.21.15.112:0/4126935319' entity='mgr.x' 2024-04-24T23:05:20.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:19 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1076962912' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T23:05:20.279 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: ignoring --setuser ceph since I am not root 2024-04-24T23:05:20.279 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: ignoring --setgroup ceph since I am not root 2024-04-24T23:05:20.279 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:20.096+0000 7f63ae159200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:05:20.582 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:20.280+0000 7f63ae159200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:05:20.582 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:20.384+0000 7f63ae159200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:05:20.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.302+0000 7fc07b062700 1 Processor -- start 2024-04-24T23:05:20.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.302+0000 7fc07b062700 1 -- start start 2024-04-24T23:05:20.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.302+0000 7fc07b062700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07407c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.302+0000 7fc07b062700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc07407c850 con 0x7fc07407de20 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.302+0000 7fc078dfe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07407c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.302+0000 7fc078dfe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07407c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33596/0 (socket says 172.21.15.112:33596) 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.302+0000 7fc078dfe700 1 -- 172.21.15.112:0/1305752272 learned_addr learned my addr 172.21.15.112:0/1305752272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.303+0000 7fc078dfe700 1 -- 172.21.15.112:0/1305752272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07407c9d0 con 0x7fc07407de20 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.303+0000 7fc078dfe700 1 --2- 172.21.15.112:0/1305752272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07407c280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0680098a0 tx=0x7fc068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0d52e5ab47c51e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.303+0000 7fc0737fe700 1 -- 172.21.15.112:0/1305752272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc068004160 con 0x7fc07407de20 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.303+0000 7fc0737fe700 1 -- 172.21.15.112:0/1305752272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc068004300 con 0x7fc07407de20 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.303+0000 7fc0737fe700 1 -- 172.21.15.112:0/1305752272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc068011bb0 con 0x7fc07407de20 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.304+0000 7fc07b062700 1 -- 172.21.15.112:0/1305752272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 msgr2=0x7fc07407c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.304+0000 7fc07b062700 1 --2- 172.21.15.112:0/1305752272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07407c280 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc0680098a0 tx=0x7fc068009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.304+0000 7fc07b062700 1 -- 172.21.15.112:0/1305752272 shutdown_connections 2024-04-24T23:05:20.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.304+0000 7fc07b062700 1 --2- 172.21.15.112:0/1305752272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07407c280 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.304+0000 7fc07b062700 1 -- 172.21.15.112:0/1305752272 >> 172.21.15.112:0/1305752272 conn(0x7fc0741032a0 msgr2=0x7fc0741056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.304+0000 7fc07b062700 1 -- 172.21.15.112:0/1305752272 shutdown_connections 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.304+0000 7fc07b062700 1 -- 172.21.15.112:0/1305752272 wait complete. 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc07b062700 1 Processor -- start 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc07b062700 1 -- start start 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc07b062700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc07b062700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc07410e380 con 0x7fc07407de20 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc078dfe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc078dfe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07419ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33598/0 (socket says 172.21.15.112:33598) 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc078dfe700 1 -- 172.21.15.112:0/1076962912 learned_addr learned my addr 172.21.15.112:0/1076962912 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc078dfe700 1 -- 172.21.15.112:0/1076962912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07419e410 con 0x7fc07407de20 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc078dfe700 1 --2- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07419ded0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc068006af0 tx=0x7fc068011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc071ffb700 1 -- 172.21.15.112:0/1076962912 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc068003af0 con 0x7fc07407de20 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc071ffb700 1 -- 172.21.15.112:0/1076962912 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc068003c90 con 0x7fc07407de20 2024-04-24T23:05:20.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc071ffb700 1 -- 172.21.15.112:0/1076962912 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc068024900 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.305+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07419e6a0 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.306+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07419eb00 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.307+0000 7fc071ffb700 1 -- 172.21.15.112:0/1076962912 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc068022020 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.307+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc074063890 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.307+0000 7fc071ffb700 1 --2- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fc064039050 0x7fc06403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.307+0000 7fc071ffb700 1 -- 172.21.15.112:0/1076962912 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc06804d120 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.307+0000 7fc073fff700 1 --2- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fc064039050 0x7fc06403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.307+0000 7fc073fff700 1 --2- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fc064039050 0x7fc06403b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc060006fd0 tx=0x7fc060006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.312+0000 7fc071ffb700 1 -- 172.21.15.112:0/1076962912 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc068017ca0 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.611+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc0741a1590 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.920+0000 7fc071ffb700 1 -- 172.21.15.112:0/1076962912 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fc06802ad60 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.922+0000 7fc071ffb700 1 -- 172.21.15.112:0/1076962912 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc06801a430 con 0x7fc07407de20 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fc064039050 msgr2=0x7fc06403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.592 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 --2- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fc064039050 0x7fc06403b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc060006fd0 tx=0x7fc060006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:20.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 msgr2=0x7fc07419ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 --2- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07419ded0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc068006af0 tx=0x7fc068011c30 comp rx=0 tx=0).stop 2024-04-24T23:05:20.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 shutdown_connections 2024-04-24T23:05:20.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 --2- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fc064039050 0x7fc06403b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 --2- 172.21.15.112:0/1076962912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc07407de20 0x7fc07419ded0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 >> 172.21.15.112:0/1076962912 conn(0x7fc0741032a0 msgr2=0x7fc074103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:20.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 shutdown_connections 2024-04-24T23:05:20.593 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:19.924+0000 7fc07b062700 1 -- 172.21.15.112:0/1076962912 wait complete. 2024-04-24T23:05:20.832 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:20.699+0000 7f63ae159200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:05:20.833 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:20.785+0000 7f63ae159200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:05:21.162 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:20 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1076962912' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T23:05:21.162 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:20 smithi112 ceph-mon[29186]: mgrmap e9: x(active, since 19s) 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.026+0000 7ff7281cc700 1 Processor -- start 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.026+0000 7ff7281cc700 1 -- start start 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.026+0000 7ff7281cc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff7201069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.941 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.026+0000 7ff7281cc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff72007bdb0 con 0x7ff720106590 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.026+0000 7ff725f68700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff7201069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.026+0000 7ff725f68700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff7201069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56736/0 (socket says 172.21.15.112:56736) 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.026+0000 7ff725f68700 1 -- 172.21.15.112:0/833512637 learned_addr learned my addr 172.21.15.112:0/833512637 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.027+0000 7ff725f68700 1 -- 172.21.15.112:0/833512637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff720106ef0 con 0x7ff720106590 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.027+0000 7ff725f68700 1 --2- 172.21.15.112:0/833512637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff7201069b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0098a0 tx=0x7ff71c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608a529cb043a202 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.027+0000 7ff724f66700 1 -- 172.21.15.112:0/833512637 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff71c004160 con 0x7ff720106590 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.027+0000 7ff724f66700 1 -- 172.21.15.112:0/833512637 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff71c004300 con 0x7ff720106590 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.027+0000 7ff724f66700 1 -- 172.21.15.112:0/833512637 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff71c011bb0 con 0x7ff720106590 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.027+0000 7ff7281cc700 1 -- 172.21.15.112:0/833512637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 msgr2=0x7ff7201069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.027+0000 7ff7281cc700 1 --2- 172.21.15.112:0/833512637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff7201069b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0098a0 tx=0x7ff71c009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.028+0000 7ff7281cc700 1 -- 172.21.15.112:0/833512637 shutdown_connections 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.028+0000 7ff7281cc700 1 --2- 172.21.15.112:0/833512637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff7201069b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.942 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.028+0000 7ff7281cc700 1 -- 172.21.15.112:0/833512637 >> 172.21.15.112:0/833512637 conn(0x7ff720102170 msgr2=0x7ff720104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.028+0000 7ff7281cc700 1 -- 172.21.15.112:0/833512637 shutdown_connections 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.028+0000 7ff7281cc700 1 -- 172.21.15.112:0/833512637 wait complete. 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.028+0000 7ff7281cc700 1 Processor -- start 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff7281cc700 1 -- start start 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff7281cc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff72019de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff7281cc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff720108020 con 0x7ff720106590 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff725f68700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff72019de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff725f68700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff72019de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56750/0 (socket says 172.21.15.112:56750) 2024-04-24T23:05:21.943 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff725f68700 1 -- 172.21.15.112:0/2411207807 learned_addr learned my addr 172.21.15.112:0/2411207807 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff725f68700 1 -- 172.21.15.112:0/2411207807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72019e3b0 con 0x7ff720106590 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff725f68700 1 --2- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff72019de70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff71c011690 tx=0x7ff71c011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff716ffd700 1 -- 172.21.15.112:0/2411207807 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff71c003930 con 0x7ff720106590 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff716ffd700 1 -- 172.21.15.112:0/2411207807 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff71c003ad0 con 0x7ff720106590 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72019e640 con 0x7ff720106590 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.029+0000 7ff716ffd700 1 -- 172.21.15.112:0/2411207807 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff71c019730 con 0x7ff720106590 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.030+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72019eaa0 con 0x7ff720106590 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.030+0000 7ff716ffd700 1 -- 172.21.15.112:0/2411207807 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff71c0199e0 con 0x7ff720106590 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.031+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff720063890 con 0x7ff720106590 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.031+0000 7ff716ffd700 1 --2- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff70c0390b0 0x7ff70c03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.031+0000 7ff725767700 1 -- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff70c0390b0 msgr2=0x7ff70c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2319978079 2024-04-24T23:05:21.944 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.031+0000 7ff716ffd700 1 -- 172.21.15.112:0/2411207807 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff71c04d560 con 0x7ff720106590 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.031+0000 7ff725767700 1 --2- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff70c0390b0 0x7ff70c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.036+0000 7ff716ffd700 1 -- 172.21.15.112:0/2411207807 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff71c003c80 con 0x7ff720106590 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.232+0000 7ff725767700 1 -- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff70c0390b0 msgr2=0x7ff70c03b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2319978079 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.232+0000 7ff725767700 1 --2- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff70c0390b0 0x7ff70c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.335+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff720062e40 con 0x7ff720106590 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.336+0000 7ff716ffd700 1 -- 172.21.15.112:0/2411207807 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff71c02d960 con 0x7ff720106590 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff70c0390b0 msgr2=0x7ff70c03b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 --2- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff70c0390b0 0x7ff70c03b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 msgr2=0x7ff72019de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:21.945 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 --2- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff72019de70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff71c011690 tx=0x7ff71c011ca0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.946 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 shutdown_connections 2024-04-24T23:05:21.946 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 --2- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7ff70c0390b0 0x7ff70c03b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.946 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 --2- 172.21.15.112:0/2411207807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff720106590 0x7ff72019de70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.946 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 >> 172.21.15.112:0/2411207807 conn(0x7ff720102170 msgr2=0x7ff720194a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:21.946 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 shutdown_connections 2024-04-24T23:05:21.946 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:21.338+0000 7ff7281cc700 1 -- 172.21.15.112:0/2411207807 wait complete. 2024-04-24T23:05:21.946 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-04-24T23:05:21.946 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 9... 2024-04-24T23:05:22.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:21 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2411207807' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:05:22.278 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:22.007+0000 7f63ae159200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:05:22.279 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:22.184+0000 7f63ae159200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:05:22.645 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:22.359+0000 7f63ae159200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:05:22.645 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:22.643+0000 7f63ae159200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:05:22.919 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:22.734+0000 7f63ae159200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:05:23.278 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:22.932+0000 7f63ae159200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:05:23.279 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:23 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:23.123+0000 7f63ae159200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:05:24.029 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:23 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:23.712+0000 7f63ae159200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:05:24.423 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:24.129+0000 7f63ae159200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:05:24.423 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:24.224+0000 7f63ae159200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:05:24.423 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:24.422+0000 7f63ae159200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:05:25.264 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:24.934+0000 7f63ae159200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:05:25.264 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:25.026+0000 7f63ae159200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:05:25.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:25.263+0000 7f63ae159200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:05:25.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:25.395+0000 7f63ae159200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:05:25.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:25.489+0000 7f63ae159200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:05:26.028 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:25.603+0000 7f63ae159200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:05:26.029 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:25.693+0000 7f63ae159200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:05:26.754 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:26.488+0000 7f63ae159200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:05:26.754 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:05:26.700+0000 7f63ae159200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: Active manager daemon x restarted 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: Activating manager daemon x 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: mgrmap e10: x(active, starting, since 0.00323689s) 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: Manager daemon x is now available 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:05:28.005 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-24T23:05:28.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:27 smithi112 ceph-mon[29186]: [24/Apr/2024:23:05:26] ENGINE Bus STARTING 2024-04-24T23:05:28.006 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:27 smithi112 ceph-mon[29186]: mgrmap e11: x(active, since 1.00561s) 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.310+0000 7fdf3af71700 1 Processor -- start 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.310+0000 7fdf3af71700 1 -- start start 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf3af71700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3410ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf3af71700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3407bdf0 con 0x7fdf34108780 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf38d0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3410ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf38d0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3410ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56760/0 (socket says 172.21.15.112:56760) 2024-04-24T23:05:28.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf38d0d700 1 -- 172.21.15.112:0/3736567729 learned_addr learned my addr 172.21.15.112:0/3736567729 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf38d0d700 1 -- 172.21.15.112:0/3736567729 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3410b0b0 con 0x7fdf34108780 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf38d0d700 1 --2- 172.21.15.112:0/3736567729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3410ab70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf280098a0 tx=0x7fdf28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104b758c52f7327b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf337fe700 1 -- 172.21.15.112:0/3736567729 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf28004160 con 0x7fdf34108780 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf337fe700 1 -- 172.21.15.112:0/3736567729 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf28004300 con 0x7fdf34108780 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.311+0000 7fdf337fe700 1 -- 172.21.15.112:0/3736567729 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf28011bb0 con 0x7fdf34108780 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.312+0000 7fdf3af71700 1 -- 172.21.15.112:0/3736567729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 msgr2=0x7fdf3410ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.312+0000 7fdf3af71700 1 --2- 172.21.15.112:0/3736567729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3410ab70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf280098a0 tx=0x7fdf28009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.312+0000 7fdf3af71700 1 -- 172.21.15.112:0/3736567729 shutdown_connections 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.312+0000 7fdf3af71700 1 --2- 172.21.15.112:0/3736567729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3410ab70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.312+0000 7fdf3af71700 1 -- 172.21.15.112:0/3736567729 >> 172.21.15.112:0/3736567729 conn(0x7fdf34102190 msgr2=0x7fdf341045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.312+0000 7fdf3af71700 1 -- 172.21.15.112:0/3736567729 shutdown_connections 2024-04-24T23:05:28.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.312+0000 7fdf3af71700 1 -- 172.21.15.112:0/3736567729 wait complete. 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.313+0000 7fdf3af71700 1 Processor -- start 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.313+0000 7fdf3af71700 1 -- start start 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.313+0000 7fdf3af71700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.313+0000 7fdf3af71700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3410c170 con 0x7fdf34108780 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.313+0000 7fdf38d0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.313+0000 7fdf38d0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56772/0 (socket says 172.21.15.112:56772) 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.313+0000 7fdf38d0d700 1 -- 172.21.15.112:0/3550697407 learned_addr learned my addr 172.21.15.112:0/3550697407 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.313+0000 7fdf38d0d700 1 -- 172.21.15.112:0/3550697407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3419c1d0 con 0x7fdf34108780 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.314+0000 7fdf38d0d700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3419bc90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf28011690 tx=0x7fdf28011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.314+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf28003930 con 0x7fdf34108780 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.314+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdf28003ad0 con 0x7fdf34108780 2024-04-24T23:05:28.286 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.314+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3419c460 con 0x7fdf34108780 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.314+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf28019730 con 0x7fdf34108780 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.314+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3419c8c0 con 0x7fdf34108780 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.315+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fdf28019a30 con 0x7fdf34108780 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.315+0000 7fdf31ffb700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 0x7fdf2403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.315+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 --> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdf2403bc20 con 0x7fdf24039050 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.315+0000 7fdf33fff700 1 -- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 msgr2=0x7fdf2403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2319978079 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.315+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf2804d670 con 0x7fdf34108780 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.315+0000 7fdf33fff700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 0x7fdf2403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.516+0000 7fdf33fff700 1 -- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 msgr2=0x7fdf2403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2319978079 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.516+0000 7fdf33fff700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 0x7fdf2403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.917+0000 7fdf33fff700 1 -- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 msgr2=0x7fdf2403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2319978079 2024-04-24T23:05:28.287 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:22.917+0000 7fdf33fff700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 0x7fdf2403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:23.719+0000 7fdf33fff700 1 -- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 msgr2=0x7fdf2403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2319978079 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:23.719+0000 7fdf33fff700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 0x7fdf2403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:25.320+0000 7fdf33fff700 1 -- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 msgr2=0x7fdf2403b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/2319978079 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:25.320+0000 7fdf33fff700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 0x7fdf2403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:26.704+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fdf2802d960 con 0x7fdf34108780 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:26.704+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 msgr2=0x7fdf2403b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:26.704+0000 7fdf31ffb700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 0x7fdf2403b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.705+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fdf28019d20 con 0x7fdf34108780 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.705+0000 7fdf31ffb700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdf2403cb10 0x7fdf2403ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.705+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdf28001d30 con 0x7fdf2403cb10 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.705+0000 7fdf33fff700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdf2403cb10 0x7fdf2403ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:28.288 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.706+0000 7fdf33fff700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdf2403cb10 0x7fdf2403ef00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf20003a10 tx=0x7fdf200092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.707+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fdf28001d30 con 0x7fdf2403cb10 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.713+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fdf34063890 con 0x7fdf2403cb10 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.713+0000 7fdf31ffb700 1 -- 172.21.15.112:0/3550697407 <== mgr.14150 v2:172.21.15.112:6800/2191305617 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fdf34063890 con 0x7fdf2403cb10 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdf2403cb10 msgr2=0x7fdf2403ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdf2403cb10 0x7fdf2403ef00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdf20003a10 tx=0x7fdf200092b0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 msgr2=0x7fdf3419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3419bc90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf28011690 tx=0x7fdf28011ca0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 shutdown_connections 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf34108780 0x7fdf3419bc90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdf2403cb10 0x7fdf2403ef00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 --2- 172.21.15.112:0/3550697407 >> [v2:172.21.15.112:6800/2319978079,v1:172.21.15.112:6801/2319978079] conn(0x7fdf24039050 0x7fdf2403b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:28.289 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 >> 172.21.15.112:0/3550697407 conn(0x7fdf34102190 msgr2=0x7fdf34102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:28.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 shutdown_connections 2024-04-24T23:05:28.290 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:27.714+0000 7fdf3af71700 1 -- 172.21.15.112:0/3550697407 wait complete. 2024-04-24T23:05:28.290 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 9 is available 2024-04-24T23:05:28.290 INFO:teuthology.orchestra.run.smithi112.stdout:Generating a dashboard self-signed certificate... 2024-04-24T23:05:28.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:28 smithi112 ceph-mon[29186]: [24/Apr/2024:23:05:27] ENGINE Serving on http://172.21.15.112:8765 2024-04-24T23:05:28.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:28 smithi112 ceph-mon[29186]: [24/Apr/2024:23:05:27] ENGINE Serving on https://172.21.15.112:7150 2024-04-24T23:05:28.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:28 smithi112 ceph-mon[29186]: [24/Apr/2024:23:05:27] ENGINE Bus STARTED 2024-04-24T23:05:28.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:28 smithi112 ceph-mon[29186]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:05:28.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:28 smithi112 ceph-mon[29186]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:05:28.943 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:29.871 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.968+0000 7faf30f09700 1 Processor -- start 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.968+0000 7faf30f09700 1 -- start start 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.968+0000 7faf30f09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c108560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.968+0000 7faf30f09700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c108b30 con 0x7faf2c108140 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.969+0000 7faf2a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.969+0000 7faf2a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56850/0 (socket says 172.21.15.112:56850) 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.969+0000 7faf2a59c700 1 -- 172.21.15.112:0/254762370 learned_addr learned my addr 172.21.15.112:0/254762370 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.969+0000 7faf2a59c700 1 -- 172.21.15.112:0/254762370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c106da0 con 0x7faf2c108140 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.969+0000 7faf2a59c700 1 --2- 172.21.15.112:0/254762370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c108560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf200098a0 tx=0x7faf20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0608c5fa3c0d856 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.969+0000 7faf2959a700 1 -- 172.21.15.112:0/254762370 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf20004160 con 0x7faf2c108140 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.969+0000 7faf2959a700 1 -- 172.21.15.112:0/254762370 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faf20004300 con 0x7faf2c108140 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.969+0000 7faf2959a700 1 -- 172.21.15.112:0/254762370 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf20011bb0 con 0x7faf2c108140 2024-04-24T23:05:29.872 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.970+0000 7faf30f09700 1 -- 172.21.15.112:0/254762370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 msgr2=0x7faf2c108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.970+0000 7faf30f09700 1 --2- 172.21.15.112:0/254762370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c108560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf200098a0 tx=0x7faf20009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.970+0000 7faf30f09700 1 -- 172.21.15.112:0/254762370 shutdown_connections 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.970+0000 7faf30f09700 1 --2- 172.21.15.112:0/254762370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c108560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.970+0000 7faf30f09700 1 -- 172.21.15.112:0/254762370 >> 172.21.15.112:0/254762370 conn(0x7faf2c101930 msgr2=0x7faf2c103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.970+0000 7faf30f09700 1 -- 172.21.15.112:0/254762370 shutdown_connections 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.970+0000 7faf30f09700 1 -- 172.21.15.112:0/254762370 wait complete. 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.971+0000 7faf30f09700 1 Processor -- start 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.971+0000 7faf30f09700 1 -- start start 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.971+0000 7faf30f09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.971+0000 7faf30f09700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf2c10b160 con 0x7faf2c108140 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.971+0000 7faf2a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.971+0000 7faf2a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56860/0 (socket says 172.21.15.112:56860) 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.971+0000 7faf2a59c700 1 -- 172.21.15.112:0/4056582989 learned_addr learned my addr 172.21.15.112:0/4056582989 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:29.873 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.972+0000 7faf2a59c700 1 -- 172.21.15.112:0/4056582989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf2c19c250 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.972+0000 7faf2a59c700 1 --2- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c19bd10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf20000c00 tx=0x7faf20011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.972+0000 7faf1b7fe700 1 -- 172.21.15.112:0/4056582989 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf20011e60 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.972+0000 7faf1b7fe700 1 -- 172.21.15.112:0/4056582989 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faf200039f0 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.972+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf2c19c4e0 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.972+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf2c19c940 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.972+0000 7faf1b7fe700 1 -- 172.21.15.112:0/4056582989 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf2001a5a0 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.973+0000 7faf1b7fe700 1 -- 172.21.15.112:0/4056582989 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7faf2001a8c0 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.973+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0c005360 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.973+0000 7faf1b7fe700 1 --2- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faf14038f30 0x7faf1403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.973+0000 7faf1b7fe700 1 -- 172.21.15.112:0/4056582989 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faf2004d0f0 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.973+0000 7faf29d9b700 1 --2- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faf14038f30 0x7faf1403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.974+0000 7faf29d9b700 1 --2- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faf14038f30 0x7faf1403b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf1c006fd0 tx=0x7faf1c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:28.978+0000 7faf1b7fe700 1 -- 172.21.15.112:0/4056582989 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faf20020240 con 0x7faf2c108140 2024-04-24T23:05:29.874 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.191+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7faf0c000bf0 con 0x7faf14038f30 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.288+0000 7faf1b7fe700 1 -- 172.21.15.112:0/4056582989 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7faf0c000bf0 con 0x7faf14038f30 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faf14038f30 msgr2=0x7faf1403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 --2- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faf14038f30 0x7faf1403b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf1c006fd0 tx=0x7faf1c006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 msgr2=0x7faf2c19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 --2- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c19bd10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf20000c00 tx=0x7faf20011c30 comp rx=0 tx=0).stop 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 shutdown_connections 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 --2- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faf14038f30 0x7faf1403b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 --2- 172.21.15.112:0/4056582989 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf2c108140 0x7faf2c19bd10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 >> 172.21.15.112:0/4056582989 conn(0x7faf2c101930 msgr2=0x7faf2c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 shutdown_connections 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:29.290+0000 7faf30f09700 1 -- 172.21.15.112:0/4056582989 wait complete. 2024-04-24T23:05:29.875 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial admin user... 2024-04-24T23:05:30.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:30 smithi112 ceph-mon[29186]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:30 smithi112 ceph-mon[29186]: mgrmap e12: x(active, since 2s) 2024-04-24T23:05:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:31.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:31 smithi112 ceph-mon[29186]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:31.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:31.851 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$K1VINK5gNE2POBulzNVCYur2qeWgpivctGRyUYCxB/yZoJTAVn9cS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713999930, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T23:05:31.851 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.261+0000 7fe52bbb1700 1 Processor -- start 2024-04-24T23:05:31.851 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.261+0000 7fe52bbb1700 1 -- start start 2024-04-24T23:05:31.851 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52bbb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe524106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52bbb1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5241075c0 con 0x7fe524106bd0 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52994d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe524106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52994d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe524106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56868/0 (socket says 172.21.15.112:56868) 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52994d700 1 -- 172.21.15.112:0/2422319283 learned_addr learned my addr 172.21.15.112:0/2422319283 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52994d700 1 -- 172.21.15.112:0/2422319283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe524107dd0 con 0x7fe524106bd0 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52994d700 1 --2- 172.21.15.112:0/2422319283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe524106ff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe5200098a0 tx=0x7fe520009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f355f02b50e1ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52894b700 1 -- 172.21.15.112:0/2422319283 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe520004160 con 0x7fe524106bd0 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52894b700 1 -- 172.21.15.112:0/2422319283 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe520004300 con 0x7fe524106bd0 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.262+0000 7fe52894b700 1 -- 172.21.15.112:0/2422319283 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe520011bb0 con 0x7fe524106bd0 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.263+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2422319283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 msgr2=0x7fe524106ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.263+0000 7fe52bbb1700 1 --2- 172.21.15.112:0/2422319283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe524106ff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe5200098a0 tx=0x7fe520009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.263+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2422319283 shutdown_connections 2024-04-24T23:05:31.852 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.263+0000 7fe52bbb1700 1 --2- 172.21.15.112:0/2422319283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe524106ff0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.263+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2422319283 >> 172.21.15.112:0/2422319283 conn(0x7fe524102190 msgr2=0x7fe5241045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.263+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2422319283 shutdown_connections 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.263+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2422319283 wait complete. 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.264+0000 7fe52bbb1700 1 Processor -- start 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.264+0000 7fe52bbb1700 1 -- start start 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.264+0000 7fe52bbb1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe52419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.264+0000 7fe52bbb1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe52410b180 con 0x7fe524106bd0 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.264+0000 7fe52994d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe52419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.264+0000 7fe52994d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe52419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56872/0 (socket says 172.21.15.112:56872) 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.264+0000 7fe52994d700 1 -- 172.21.15.112:0/2198252109 learned_addr learned my addr 172.21.15.112:0/2198252109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.264+0000 7fe52994d700 1 -- 172.21.15.112:0/2198252109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52419c270 con 0x7fe524106bd0 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.265+0000 7fe52994d700 1 --2- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe52419bd30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe520009520 tx=0x7fe520011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.265+0000 7fe516ffd700 1 -- 172.21.15.112:0/2198252109 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe520011e60 con 0x7fe524106bd0 2024-04-24T23:05:31.853 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.265+0000 7fe516ffd700 1 -- 172.21.15.112:0/2198252109 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe5200039c0 con 0x7fe524106bd0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.265+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52419c500 con 0x7fe524106bd0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.265+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52419c960 con 0x7fe524106bd0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.265+0000 7fe516ffd700 1 -- 172.21.15.112:0/2198252109 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe52001a5a0 con 0x7fe524106bd0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.266+0000 7fe516ffd700 1 -- 172.21.15.112:0/2198252109 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe52001a880 con 0x7fe524106bd0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.266+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe524063890 con 0x7fe524106bd0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.266+0000 7fe516ffd700 1 --2- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5100390a0 0x7fe51003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.266+0000 7fe516ffd700 1 -- 172.21.15.112:0/2198252109 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe52004d490 con 0x7fe524106bd0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.266+0000 7fe52914c700 1 --2- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5100390a0 0x7fe51003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.266+0000 7fe52914c700 1 --2- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5100390a0 0x7fe51003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe518006fd0 tx=0x7fe518006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.271+0000 7fe516ffd700 1 -- 172.21.15.112:0/2198252109 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe52000fdd0 con 0x7fe524106bd0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.498+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fe524195d40 con 0x7fe5100390a0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.723+0000 7fe516ffd700 1 -- 172.21.15.112:0/2198252109 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fe524195d40 con 0x7fe5100390a0 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5100390a0 msgr2=0x7fe51003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.854 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 --2- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5100390a0 0x7fe51003b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe518006fd0 tx=0x7fe518006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 msgr2=0x7fe52419bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 --2- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe52419bd30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe520009520 tx=0x7fe520011c30 comp rx=0 tx=0).stop 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 shutdown_connections 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 --2- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5100390a0 0x7fe51003b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 --2- 172.21.15.112:0/2198252109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe524106bd0 0x7fe52419bd30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 >> 172.21.15.112:0/2198252109 conn(0x7fe524102190 msgr2=0x7fe524103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 shutdown_connections 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:30.725+0000 7fe52bbb1700 1 -- 172.21.15.112:0/2198252109 wait complete. 2024-04-24T23:05:31.855 INFO:teuthology.orchestra.run.smithi112.stdout:Fetching dashboard port number... 2024-04-24T23:05:33.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:32 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3546606342' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T23:05:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T23:05:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.268+0000 7eff245a6700 1 Processor -- start 2024-04-24T23:05:33.064 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.268+0000 7eff245a6700 1 -- start start 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.268+0000 7eff245a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.268+0000 7eff245a6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c106980 con 0x7eff1c105f90 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.268+0000 7eff22342700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.268+0000 7eff22342700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52574/0 (socket says 172.21.15.112:52574) 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.268+0000 7eff22342700 1 -- 172.21.15.112:0/4259015199 learned_addr learned my addr 172.21.15.112:0/4259015199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.268+0000 7eff22342700 1 -- 172.21.15.112:0/4259015199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c108f20 con 0x7eff1c105f90 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff22342700 1 --2- 172.21.15.112:0/4259015199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1063b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff180098a0 tx=0x7eff18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e76c1f513971f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff21340700 1 -- 172.21.15.112:0/4259015199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff18004160 con 0x7eff1c105f90 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff21340700 1 -- 172.21.15.112:0/4259015199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff18004300 con 0x7eff1c105f90 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff21340700 1 -- 172.21.15.112:0/4259015199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff18011bb0 con 0x7eff1c105f90 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff245a6700 1 -- 172.21.15.112:0/4259015199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 msgr2=0x7eff1c1063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff245a6700 1 --2- 172.21.15.112:0/4259015199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1063b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff180098a0 tx=0x7eff18009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.065 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff245a6700 1 -- 172.21.15.112:0/4259015199 shutdown_connections 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff245a6700 1 --2- 172.21.15.112:0/4259015199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1063b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff245a6700 1 -- 172.21.15.112:0/4259015199 >> 172.21.15.112:0/4259015199 conn(0x7eff1c101900 msgr2=0x7eff1c103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff245a6700 1 -- 172.21.15.112:0/4259015199 shutdown_connections 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.269+0000 7eff245a6700 1 -- 172.21.15.112:0/4259015199 wait complete. 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.270+0000 7eff245a6700 1 Processor -- start 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.270+0000 7eff245a6700 1 -- start start 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.270+0000 7eff245a6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1087a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.270+0000 7eff245a6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c10c2d0 con 0x7eff1c105f90 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.270+0000 7eff22342700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.270+0000 7eff22342700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52588/0 (socket says 172.21.15.112:52588) 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.270+0000 7eff22342700 1 -- 172.21.15.112:0/3546606342 learned_addr learned my addr 172.21.15.112:0/3546606342 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.270+0000 7eff22342700 1 -- 172.21.15.112:0/3546606342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c108ce0 con 0x7eff1c105f90 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.271+0000 7eff22342700 1 --2- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1087a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff18011690 tx=0x7eff18011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:33.066 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.271+0000 7eff137fe700 1 -- 172.21.15.112:0/3546606342 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff18003b70 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.271+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c106ed0 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.271+0000 7eff137fe700 1 -- 172.21.15.112:0/3546606342 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff1801a930 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.271+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c107390 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.271+0000 7eff137fe700 1 -- 172.21.15.112:0/3546606342 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff18019920 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.272+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1c063890 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.272+0000 7eff137fe700 1 -- 172.21.15.112:0/3546606342 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7eff18022020 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.272+0000 7eff137fe700 1 --2- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff08039050 0x7eff0803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.272+0000 7eff137fe700 1 -- 172.21.15.112:0/3546606342 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff1804d3f0 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.272+0000 7eff21b41700 1 --2- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff08039050 0x7eff0803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.272+0000 7eff21b41700 1 --2- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff08039050 0x7eff0803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff0c006fd0 tx=0x7eff0c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.277+0000 7eff137fe700 1 -- 172.21.15.112:0/3546606342 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff1800ed50 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.473+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7eff1c107680 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.474+0000 7eff137fe700 1 -- 172.21.15.112:0/3546606342 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7eff1c107680 con 0x7eff1c105f90 2024-04-24T23:05:33.067 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff08039050 msgr2=0x7eff0803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 --2- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff08039050 0x7eff0803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7eff0c006fd0 tx=0x7eff0c006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 msgr2=0x7eff1c1087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 --2- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1087a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7eff18011690 tx=0x7eff18011ca0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 shutdown_connections 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 --2- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff08039050 0x7eff0803b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 --2- 172.21.15.112:0/3546606342 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff1c105f90 0x7eff1c1087a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 >> 172.21.15.112:0/3546606342 conn(0x7eff1c101900 msgr2=0x7eff1c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 shutdown_connections 2024-04-24T23:05:33.068 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:32.476+0000 7eff245a6700 1 -- 172.21.15.112:0/3546606342 wait complete. 2024-04-24T23:05:33.071 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:05:33.071 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-24T23:05:33.077 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:05:33.077 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-24T23:05:33.077 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-24T23:05:33.077 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T23:05:33.079 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph Dashboard is now available at: 2024-04-24T23:05:33.079 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:33.079 INFO:teuthology.orchestra.run.smithi112.stdout: URL: https://smithi112:8443/ 2024-04-24T23:05:33.079 INFO:teuthology.orchestra.run.smithi112.stdout: User: admin 2024-04-24T23:05:33.079 INFO:teuthology.orchestra.run.smithi112.stdout: Password: kh9p2sbepg 2024-04-24T23:05:33.079 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:33.080 INFO:teuthology.orchestra.run.smithi112.stdout:Saving cluster configuration to /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config directory 2024-04-24T23:05:33.080 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling autotune for osd_memory_target 2024-04-24T23:05:33.929 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:33 smithi112 ceph-mon[29186]: mgrmap e13: x(active, since 6s) 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.442+0000 7fe61aa30700 1 Processor -- start 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.442+0000 7fe61aa30700 1 -- start start 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.442+0000 7fe61aa30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe614108580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.442+0000 7fe61aa30700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe614108b50 con 0x7fe614108160 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.442+0000 7fe613fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe614108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.443+0000 7fe613fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe614108580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52600/0 (socket says 172.21.15.112:52600) 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.443+0000 7fe613fff700 1 -- 172.21.15.112:0/1108769421 learned_addr learned my addr 172.21.15.112:0/1108769421 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.443+0000 7fe613fff700 1 -- 172.21.15.112:0/1108769421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe614106d70 con 0x7fe614108160 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.443+0000 7fe613fff700 1 --2- 172.21.15.112:0/1108769421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe614108580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6040098a0 tx=0x7fe604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8981ae0c1b8ecc0f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.443+0000 7fe612ffd700 1 -- 172.21.15.112:0/1108769421 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604004160 con 0x7fe614108160 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.443+0000 7fe612ffd700 1 -- 172.21.15.112:0/1108769421 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe604004300 con 0x7fe614108160 2024-04-24T23:05:34.171 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.443+0000 7fe612ffd700 1 -- 172.21.15.112:0/1108769421 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604011bb0 con 0x7fe614108160 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 -- 172.21.15.112:0/1108769421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 msgr2=0x7fe614108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 --2- 172.21.15.112:0/1108769421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe614108580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6040098a0 tx=0x7fe604009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 -- 172.21.15.112:0/1108769421 shutdown_connections 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 --2- 172.21.15.112:0/1108769421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe614108580 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 -- 172.21.15.112:0/1108769421 >> 172.21.15.112:0/1108769421 conn(0x7fe6141018c0 msgr2=0x7fe614103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 -- 172.21.15.112:0/1108769421 shutdown_connections 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 -- 172.21.15.112:0/1108769421 wait complete. 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 Processor -- start 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.444+0000 7fe61aa30700 1 -- start start 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe61aa30700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe6141066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe61aa30700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe61410b130 con 0x7fe614108160 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe613fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe6141066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe613fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe6141066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52606/0 (socket says 172.21.15.112:52606) 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe613fff700 1 -- 172.21.15.112:0/607838594 learned_addr learned my addr 172.21.15.112:0/607838594 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:34.172 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe613fff700 1 -- 172.21.15.112:0/607838594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61419ea30 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe613fff700 1 --2- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe6141066e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe604009520 tx=0x7fe604011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe6117fa700 1 -- 172.21.15.112:0/607838594 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe604011e60 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe6117fa700 1 -- 172.21.15.112:0/607838594 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6040039c0 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61419ec00 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe6117fa700 1 -- 172.21.15.112:0/607838594 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe60401a5a0 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.445+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61419efa0 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.446+0000 7fe6117fa700 1 -- 172.21.15.112:0/607838594 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe60401a810 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.446+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61410b2b0 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.447+0000 7fe6117fa700 1 --2- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5fc0391a0 0x7fe5fc03b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.447+0000 7fe6117fa700 1 -- 172.21.15.112:0/607838594 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe60404d460 con 0x7fe614108160 2024-04-24T23:05:34.173 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.447+0000 7fe6137fe700 1 --2- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5fc0391a0 0x7fe5fc03b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.447+0000 7fe6137fe700 1 --2- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5fc0391a0 0x7fe5fc03b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe608006fd0 tx=0x7fe608006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.451+0000 7fe6117fa700 1 -- 172.21.15.112:0/607838594 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe604019ca0 con 0x7fe614108160 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.646+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe61419f460 con 0x7fe614108160 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.646+0000 7fe6117fa700 1 -- 172.21.15.112:0/607838594 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe6040042c0 con 0x7fe614108160 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5fc0391a0 msgr2=0x7fe5fc03b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 --2- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5fc0391a0 0x7fe5fc03b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe608006fd0 tx=0x7fe608006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 msgr2=0x7fe6141066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 --2- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe6141066e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe604009520 tx=0x7fe604011c30 comp rx=0 tx=0).stop 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 shutdown_connections 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 --2- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5fc0391a0 0x7fe5fc03b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 --2- 172.21.15.112:0/607838594 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe614108160 0x7fe6141066e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 >> 172.21.15.112:0/607838594 conn(0x7fe6141018c0 msgr2=0x7fe614102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 shutdown_connections 2024-04-24T23:05:34.174 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:33.648+0000 7fe61aa30700 1 -- 172.21.15.112:0/607838594 wait complete. 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.588+0000 7f149a889700 1 Processor -- start 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.588+0000 7f149a889700 1 -- start start 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f149a889700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f14941080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f149a889700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14941086b0 con 0x7f1494107cc0 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f1493fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f14941080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f1493fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f14941080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52608/0 (socket says 172.21.15.112:52608) 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f1493fff700 1 -- 172.21.15.112:0/2758621702 learned_addr learned my addr 172.21.15.112:0/2758621702 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f1493fff700 1 -- 172.21.15.112:0/2758621702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1494108ec0 con 0x7f1494107cc0 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f1493fff700 1 --2- 172.21.15.112:0/2758621702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f14941080e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14840098a0 tx=0x7f1484009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb26144db568e7c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.427 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f1492ffd700 1 -- 172.21.15.112:0/2758621702 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1484004160 con 0x7f1494107cc0 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f1492ffd700 1 -- 172.21.15.112:0/2758621702 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1484004300 con 0x7f1494107cc0 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.589+0000 7f1492ffd700 1 -- 172.21.15.112:0/2758621702 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1484011bb0 con 0x7f1494107cc0 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 -- 172.21.15.112:0/2758621702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 msgr2=0x7f14941080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 --2- 172.21.15.112:0/2758621702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f14941080e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14840098a0 tx=0x7f1484009bb0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 -- 172.21.15.112:0/2758621702 shutdown_connections 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 --2- 172.21.15.112:0/2758621702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f14941080e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 -- 172.21.15.112:0/2758621702 >> 172.21.15.112:0/2758621702 conn(0x7f14941032a0 msgr2=0x7f14941056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 -- 172.21.15.112:0/2758621702 shutdown_connections 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 -- 172.21.15.112:0/2758621702 wait complete. 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 Processor -- start 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.590+0000 7f149a889700 1 -- start start 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f149a889700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f149407daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f149a889700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f149410c270 con 0x7f1494107cc0 2024-04-24T23:05:35.428 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f1493fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f149407daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f1493fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f149407daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52622/0 (socket says 172.21.15.112:52622) 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f1493fff700 1 -- 172.21.15.112:0/1475546698 learned_addr learned my addr 172.21.15.112:0/1475546698 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f1493fff700 1 -- 172.21.15.112:0/1475546698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149407e030 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f1493fff700 1 --2- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f149407daf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1484011690 tx=0x7f1484011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f14917fa700 1 -- 172.21.15.112:0/1475546698 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14840039c0 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f14917fa700 1 -- 172.21.15.112:0/1475546698 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1484003b60 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f14917fa700 1 -- 172.21.15.112:0/1475546698 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14840197d0 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.591+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149407c220 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.592+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149407c6e0 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.592+0000 7f14917fa700 1 -- 172.21.15.112:0/1475546698 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1484019a40 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.592+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1494063890 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.592+0000 7f14917fa700 1 --2- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f147c039150 0x7f147c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.592+0000 7f14917fa700 1 -- 172.21.15.112:0/1475546698 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f148404d730 con 0x7f1494107cc0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.593+0000 7f14937fe700 1 --2- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f147c039150 0x7f147c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:35.429 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.593+0000 7f14937fe700 1 --2- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f147c039150 0x7f147c03b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1488006fd0 tx=0x7f1488006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.598+0000 7f14917fa700 1 -- 172.21.15.112:0/1475546698 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14840042a0 con 0x7f1494107cc0 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.907+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f149407c960 con 0x7f1494107cc0 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.909+0000 7f14917fa700 1 -- 172.21.15.112:0/1475546698 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f1484016db0 con 0x7f1494107cc0 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.911+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f147c039150 msgr2=0x7f147c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.911+0000 7f149a889700 1 --2- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f147c039150 0x7f147c03b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1488006fd0 tx=0x7f1488006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.911+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 msgr2=0x7f149407daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.911+0000 7f149a889700 1 --2- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f149407daf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1484011690 tx=0x7f1484011c30 comp rx=0 tx=0).stop 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.912+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 shutdown_connections 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.912+0000 7f149a889700 1 --2- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f147c039150 0x7f147c03b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.912+0000 7f149a889700 1 --2- 172.21.15.112:0/1475546698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1494107cc0 0x7f149407daf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.912+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 >> 172.21.15.112:0/1475546698 conn(0x7f14941032a0 msgr2=0x7f1494103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.912+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 shutdown_connections 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-24T23:05:34.912+0000 7f149a889700 1 -- 172.21.15.112:0/1475546698 wait complete. 2024-04-24T23:05:35.430 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: ceph telemetry on 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout:For more information see: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-24T23:05:35.431 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:35.432 INFO:teuthology.orchestra.run.smithi112.stdout:Bootstrap complete. 2024-04-24T23:05:35.461 INFO:tasks.cephadm:Fetching config... 2024-04-24T23:05:35.461 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:05:35.461 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T23:05:35.476 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T23:05:35.476 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:05:35.476 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T23:05:35.531 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T23:05:35.532 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:05:35.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-24T23:05:35.595 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T23:05:35.595 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:05:35.595 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T23:05:35.651 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T23:05:35.651 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNsvObsMR//qQxiYg0j+66E0veSpCPpxanjsMSWnVruZe2n5pwrfgoST0EvgNBaluGK1wuF+GoebwYhuBVG1/6WbZImCU7ROJvjkFXi8+fqO6IRybtlh1/fQfLz6wsnuIdkZxcY6mFyi/KrbXgHbs7whi0R3hEnqdrIaZH8aFBYDApSGzk5+q/Nk1a4yBxRaL3ZXmecdRH0uwQdn0XqD2n9xLuTum1CJCkrR0+7QgDM6NAiL6x5gdrg1ORahqSspuilOzSncKKFQso1VZid+jP0RvpFjLXafEiNcxWE5GX0LKE9i1tuOS4fDoE6S39iWJmm7JczKWfveBEZi7D8GkJ2XJajgOaVTSjqaGpAHh+7+0KqedaO0IhCs0yvTsg5Wds2uTyvDIAe+VGUbjOMi9yS+3gPgYDUzvT3kALseFRuJUFqYgPmiTVwxAKp+hLJ9DtXWhBWdpQN50rsfu72Pp9ApBIomDxeff4SIFkkoMZsaw7KjselH7dnRunFJ9e0Rc= ceph-e62ba378-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:05:35.723 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNsvObsMR//qQxiYg0j+66E0veSpCPpxanjsMSWnVruZe2n5pwrfgoST0EvgNBaluGK1wuF+GoebwYhuBVG1/6WbZImCU7ROJvjkFXi8+fqO6IRybtlh1/fQfLz6wsnuIdkZxcY6mFyi/KrbXgHbs7whi0R3hEnqdrIaZH8aFBYDApSGzk5+q/Nk1a4yBxRaL3ZXmecdRH0uwQdn0XqD2n9xLuTum1CJCkrR0+7QgDM6NAiL6x5gdrg1ORahqSspuilOzSncKKFQso1VZid+jP0RvpFjLXafEiNcxWE5GX0LKE9i1tuOS4fDoE6S39iWJmm7JczKWfveBEZi7D8GkJ2XJajgOaVTSjqaGpAHh+7+0KqedaO0IhCs0yvTsg5Wds2uTyvDIAe+VGUbjOMi9yS+3gPgYDUzvT3kALseFRuJUFqYgPmiTVwxAKp+hLJ9DtXWhBWdpQN50rsfu72Pp9ApBIomDxeff4SIFkkoMZsaw7KjselH7dnRunFJ9e0Rc= ceph-e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:05:35.733 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNsvObsMR//qQxiYg0j+66E0veSpCPpxanjsMSWnVruZe2n5pwrfgoST0EvgNBaluGK1wuF+GoebwYhuBVG1/6WbZImCU7ROJvjkFXi8+fqO6IRybtlh1/fQfLz6wsnuIdkZxcY6mFyi/KrbXgHbs7whi0R3hEnqdrIaZH8aFBYDApSGzk5+q/Nk1a4yBxRaL3ZXmecdRH0uwQdn0XqD2n9xLuTum1CJCkrR0+7QgDM6NAiL6x5gdrg1ORahqSspuilOzSncKKFQso1VZid+jP0RvpFjLXafEiNcxWE5GX0LKE9i1tuOS4fDoE6S39iWJmm7JczKWfveBEZi7D8GkJ2XJajgOaVTSjqaGpAHh+7+0KqedaO0IhCs0yvTsg5Wds2uTyvDIAe+VGUbjOMi9yS+3gPgYDUzvT3kALseFRuJUFqYgPmiTVwxAKp+hLJ9DtXWhBWdpQN50rsfu72Pp9ApBIomDxeff4SIFkkoMZsaw7KjselH7dnRunFJ9e0Rc= ceph-e62ba378-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:05:35.772 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNsvObsMR//qQxiYg0j+66E0veSpCPpxanjsMSWnVruZe2n5pwrfgoST0EvgNBaluGK1wuF+GoebwYhuBVG1/6WbZImCU7ROJvjkFXi8+fqO6IRybtlh1/fQfLz6wsnuIdkZxcY6mFyi/KrbXgHbs7whi0R3hEnqdrIaZH8aFBYDApSGzk5+q/Nk1a4yBxRaL3ZXmecdRH0uwQdn0XqD2n9xLuTum1CJCkrR0+7QgDM6NAiL6x5gdrg1ORahqSspuilOzSncKKFQso1VZid+jP0RvpFjLXafEiNcxWE5GX0LKE9i1tuOS4fDoE6S39iWJmm7JczKWfveBEZi7D8GkJ2XJajgOaVTSjqaGpAHh+7+0KqedaO0IhCs0yvTsg5Wds2uTyvDIAe+VGUbjOMi9yS+3gPgYDUzvT3kALseFRuJUFqYgPmiTVwxAKp+hLJ9DtXWhBWdpQN50rsfu72Pp9ApBIomDxeff4SIFkkoMZsaw7KjselH7dnRunFJ9e0Rc= ceph-e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:05:35.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNsvObsMR//qQxiYg0j+66E0veSpCPpxanjsMSWnVruZe2n5pwrfgoST0EvgNBaluGK1wuF+GoebwYhuBVG1/6WbZImCU7ROJvjkFXi8+fqO6IRybtlh1/fQfLz6wsnuIdkZxcY6mFyi/KrbXgHbs7whi0R3hEnqdrIaZH8aFBYDApSGzk5+q/Nk1a4yBxRaL3ZXmecdRH0uwQdn0XqD2n9xLuTum1CJCkrR0+7QgDM6NAiL6x5gdrg1ORahqSspuilOzSncKKFQso1VZid+jP0RvpFjLXafEiNcxWE5GX0LKE9i1tuOS4fDoE6S39iWJmm7JczKWfveBEZi7D8GkJ2XJajgOaVTSjqaGpAHh+7+0KqedaO0IhCs0yvTsg5Wds2uTyvDIAe+VGUbjOMi9yS+3gPgYDUzvT3kALseFRuJUFqYgPmiTVwxAKp+hLJ9DtXWhBWdpQN50rsfu72Pp9ApBIomDxeff4SIFkkoMZsaw7KjselH7dnRunFJ9e0Rc= ceph-e62ba378-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:05:35.823 INFO:teuthology.orchestra.run.smithi187.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNsvObsMR//qQxiYg0j+66E0veSpCPpxanjsMSWnVruZe2n5pwrfgoST0EvgNBaluGK1wuF+GoebwYhuBVG1/6WbZImCU7ROJvjkFXi8+fqO6IRybtlh1/fQfLz6wsnuIdkZxcY6mFyi/KrbXgHbs7whi0R3hEnqdrIaZH8aFBYDApSGzk5+q/Nk1a4yBxRaL3ZXmecdRH0uwQdn0XqD2n9xLuTum1CJCkrR0+7QgDM6NAiL6x5gdrg1ORahqSspuilOzSncKKFQso1VZid+jP0RvpFjLXafEiNcxWE5GX0LKE9i1tuOS4fDoE6S39iWJmm7JczKWfveBEZi7D8GkJ2XJajgOaVTSjqaGpAHh+7+0KqedaO0IhCs0yvTsg5Wds2uTyvDIAe+VGUbjOMi9yS+3gPgYDUzvT3kALseFRuJUFqYgPmiTVwxAKp+hLJ9DtXWhBWdpQN50rsfu72Pp9ApBIomDxeff4SIFkkoMZsaw7KjselH7dnRunFJ9e0Rc= ceph-e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:05:35.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T23:05:36.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:35 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1475546698' entity='client.admin' 2024-04-24T23:05:36.074 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:38.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.347+0000 7f3f748d8700 1 -- 172.21.15.112:0/2720617559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 msgr2=0x7f3f701025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.347+0000 7f3f748d8700 1 --2- 172.21.15.112:0/2720617559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 0x7f3f701025c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009910 tx=0x7f3f60009c20 comp rx=0 tx=0).stop 2024-04-24T23:05:38.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.347+0000 7f3f748d8700 1 -- 172.21.15.112:0/2720617559 shutdown_connections 2024-04-24T23:05:38.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.347+0000 7f3f748d8700 1 --2- 172.21.15.112:0/2720617559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 0x7f3f701025c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.347+0000 7f3f748d8700 1 -- 172.21.15.112:0/2720617559 >> 172.21.15.112:0/2720617559 conn(0x7f3f700fb8b0 msgr2=0x7f3f700fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:38.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.347+0000 7f3f748d8700 1 -- 172.21.15.112:0/2720617559 shutdown_connections 2024-04-24T23:05:38.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.347+0000 7f3f748d8700 1 -- 172.21.15.112:0/2720617559 wait complete. 2024-04-24T23:05:38.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.348+0000 7f3f748d8700 1 Processor -- start 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.348+0000 7f3f748d8700 1 -- start start 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.348+0000 7f3f748d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 0x7f3f701973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.348+0000 7f3f748d8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f70106d70 con 0x7f3f701021a0 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.348+0000 7f3f6e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 0x7f3f701973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.348+0000 7f3f6e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 0x7f3f701973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52646/0 (socket says 172.21.15.112:52646) 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.348+0000 7f3f6e59c700 1 -- 172.21.15.112:0/2609704889 learned_addr learned my addr 172.21.15.112:0/2609704889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.349+0000 7f3f6e59c700 1 -- 172.21.15.112:0/2609704889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f70197920 con 0x7f3f701021a0 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.349+0000 7f3f6e59c700 1 --2- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 0x7f3f701973e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f60005840 tx=0x7f3f600050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.349+0000 7f3f5f7fe700 1 -- 172.21.15.112:0/2609704889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f60027070 con 0x7f3f701021a0 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.349+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f70197bb0 con 0x7f3f701021a0 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.349+0000 7f3f5f7fe700 1 -- 172.21.15.112:0/2609704889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3f60022470 con 0x7f3f701021a0 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.349+0000 7f3f5f7fe700 1 -- 172.21.15.112:0/2609704889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f6000f5f0 con 0x7f3f701021a0 2024-04-24T23:05:38.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.349+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f7019a8a0 con 0x7f3f701021a0 2024-04-24T23:05:38.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.350+0000 7f3f5f7fe700 1 -- 172.21.15.112:0/2609704889 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3f6002a070 con 0x7f3f701021a0 2024-04-24T23:05:38.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.350+0000 7f3f5f7fe700 1 --2- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3f580390f0 0x7f3f5803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:38.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.350+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f701074f0 con 0x7f3f701021a0 2024-04-24T23:05:38.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.350+0000 7f3f5f7fe700 1 -- 172.21.15.112:0/2609704889 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f6001e030 con 0x7f3f701021a0 2024-04-24T23:05:38.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.350+0000 7f3f6dd9b700 1 --2- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3f580390f0 0x7f3f5803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:38.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.350+0000 7f3f6dd9b700 1 --2- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3f580390f0 0x7f3f5803b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3f64006fd0 tx=0x7f3f64006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:38.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.355+0000 7f3f5f7fe700 1 -- 172.21.15.112:0/2609704889 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f60013070 con 0x7f3f701021a0 2024-04-24T23:05:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.557+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3f7002dc40 con 0x7f3f701021a0 2024-04-24T23:05:38.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.559+0000 7f3f5f7fe700 1 -- 172.21.15.112:0/2609704889 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3f60028920 con 0x7f3f701021a0 2024-04-24T23:05:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3f580390f0 msgr2=0x7f3f5803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 --2- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3f580390f0 0x7f3f5803b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3f64006fd0 tx=0x7f3f64006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:38.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 msgr2=0x7f3f701973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 --2- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 0x7f3f701973e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f60005840 tx=0x7f3f600050b0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 shutdown_connections 2024-04-24T23:05:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 --2- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3f580390f0 0x7f3f5803b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 --2- 172.21.15.112:0/2609704889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f701021a0 0x7f3f701973e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 >> 172.21.15.112:0/2609704889 conn(0x7f3f700fb8b0 msgr2=0x7f3f70100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.561+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 shutdown_connections 2024-04-24T23:05:38.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:38.562+0000 7f3f748d8700 1 -- 172.21.15.112:0/2609704889 wait complete. 2024-04-24T23:05:39.254 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T23:05:39.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T23:05:39.930 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:39 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2609704889' entity='client.admin' 2024-04-24T23:05:39.958 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:41.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:41.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:41.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:41.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:41.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:41.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 -- 172.21.15.112:0/3668632642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 msgr2=0x7f2eb81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 --2- 172.21.15.112:0/3668632642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 0x7f2eb81020b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009910 tx=0x7f2eac009c20 comp rx=0 tx=0).stop 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 -- 172.21.15.112:0/3668632642 shutdown_connections 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 --2- 172.21.15.112:0/3668632642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 0x7f2eb81020b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 -- 172.21.15.112:0/3668632642 >> 172.21.15.112:0/3668632642 conn(0x7f2eb80fd250 msgr2=0x7f2eb80ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 -- 172.21.15.112:0/3668632642 shutdown_connections 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 -- 172.21.15.112:0/3668632642 wait complete. 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 Processor -- start 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 -- start start 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 0x7f2eb8075ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.360+0000 7f2ebec1a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb8107e60 con 0x7f2eb8101c90 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2ebc9b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 0x7f2eb8075ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2ebc9b6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 0x7f2eb8075ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41476/0 (socket says 172.21.15.112:41476) 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2ebc9b6700 1 -- 172.21.15.112:0/2491394351 learned_addr learned my addr 172.21.15.112:0/2491394351 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2ebc9b6700 1 -- 172.21.15.112:0/2491394351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb8076430 con 0x7f2eb8101c90 2024-04-24T23:05:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2ebc9b6700 1 --2- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 0x7f2eb8075ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2eac005840 tx=0x7f2eac0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2eb5ffb700 1 -- 172.21.15.112:0/2491394351 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2eac027070 con 0x7f2eb8101c90 2024-04-24T23:05:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb8074620 con 0x7f2eb8101c90 2024-04-24T23:05:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2eb5ffb700 1 -- 172.21.15.112:0/2491394351 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2eac022470 con 0x7f2eb8101c90 2024-04-24T23:05:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2eb5ffb700 1 -- 172.21.15.112:0/2491394351 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2eac00f5f0 con 0x7f2eb8101c90 2024-04-24T23:05:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.361+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb8074b60 con 0x7f2eb8101c90 2024-04-24T23:05:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.362+0000 7f2eb5ffb700 1 -- 172.21.15.112:0/2491394351 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f2eac02a070 con 0x7f2eb8101c90 2024-04-24T23:05:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.362+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb81085e0 con 0x7f2eb8101c90 2024-04-24T23:05:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.363+0000 7f2eb5ffb700 1 --2- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2ea80390f0 0x7f2ea803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.363+0000 7f2eb5ffb700 1 -- 172.21.15.112:0/2491394351 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2eac01e030 con 0x7f2eb8101c90 2024-04-24T23:05:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.363+0000 7f2eb7fff700 1 --2- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2ea80390f0 0x7f2ea803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.363+0000 7f2eb7fff700 1 --2- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2ea80390f0 0x7f2ea803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4006fd0 tx=0x7f2ea4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.368+0000 7f2eb5ffb700 1 -- 172.21.15.112:0/2491394351 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2eac013070 con 0x7f2eb8101c90 2024-04-24T23:05:41.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.567+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f2eb8190900 con 0x7f2ea80390f0 2024-04-24T23:05:41.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.571+0000 7f2eb5ffb700 1 -- 172.21.15.112:0/2491394351 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2eb8190900 con 0x7f2ea80390f0 2024-04-24T23:05:41.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2ea80390f0 msgr2=0x7f2ea803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 --2- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2ea80390f0 0x7f2ea803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4006fd0 tx=0x7f2ea4006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 msgr2=0x7f2eb8075ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 --2- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 0x7f2eb8075ef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2eac005840 tx=0x7f2eac0050b0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 shutdown_connections 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 --2- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2ea80390f0 0x7f2ea803b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 --2- 172.21.15.112:0/2491394351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2eb8101c90 0x7f2eb8075ef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 >> 172.21.15.112:0/2491394351 conn(0x7f2eb80fd250 msgr2=0x7f2eb80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 shutdown_connections 2024-04-24T23:05:41.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:41.573+0000 7f2ebec1a700 1 -- 172.21.15.112:0/2491394351 wait complete. 2024-04-24T23:05:42.356 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-04-24T23:05:42.356 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:05:42.356 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:05:42.376 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:05:42.376 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:05:42.434 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-04-24T23:05:42.434 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch host add smithi150 2024-04-24T23:05:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:42.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:42.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:42.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:42.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:42.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:42.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-24T23:05:42.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: Updating smithi112:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:05:42.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:42.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:42.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:43.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:43 smithi112 ceph-mon[29186]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:05:43.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:43 smithi112 ceph-mon[29186]: Updating smithi112:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:05:44.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.239+0000 7f0f07387700 1 -- 172.21.15.112:0/3490695052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 msgr2=0x7f0f00102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.239+0000 7f0f07387700 1 --2- 172.21.15.112:0/3490695052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 0x7f0f00102260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009910 tx=0x7f0ef0009c20 comp rx=0 tx=0).stop 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.239+0000 7f0f07387700 1 -- 172.21.15.112:0/3490695052 shutdown_connections 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.239+0000 7f0f07387700 1 --2- 172.21.15.112:0/3490695052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 0x7f0f00102260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.239+0000 7f0f07387700 1 -- 172.21.15.112:0/3490695052 >> 172.21.15.112:0/3490695052 conn(0x7f0f000fd750 msgr2=0x7f0f000ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.239+0000 7f0f07387700 1 -- 172.21.15.112:0/3490695052 shutdown_connections 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.239+0000 7f0f07387700 1 -- 172.21.15.112:0/3490695052 wait complete. 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.240+0000 7f0f07387700 1 Processor -- start 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.240+0000 7f0f07387700 1 -- start start 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.240+0000 7f0f07387700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 0x7f0f0019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.240+0000 7f0f07387700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f0010c390 con 0x7f0f00101e80 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.240+0000 7f0f05123700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 0x7f0f0019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:44.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.240+0000 7f0f05123700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 0x7f0f0019d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41486/0 (socket says 172.21.15.112:41486) 2024-04-24T23:05:44.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.240+0000 7f0f05123700 1 -- 172.21.15.112:0/1941468405 learned_addr learned my addr 172.21.15.112:0/1941468405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:44.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.240+0000 7f0f05123700 1 -- 172.21.15.112:0/1941468405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0019d910 con 0x7f0f00101e80 2024-04-24T23:05:44.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.241+0000 7f0f05123700 1 --2- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 0x7f0f0019d3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00052a0 tx=0x7f0ef0004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:44.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.241+0000 7f0ef67fc700 1 -- 172.21.15.112:0/1941468405 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ef0026070 con 0x7f0f00101e80 2024-04-24T23:05:44.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.241+0000 7f0ef67fc700 1 -- 172.21.15.112:0/1941468405 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0ef000bbf0 con 0x7f0f00101e80 2024-04-24T23:05:44.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.241+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0019dc00 con 0x7f0f00101e80 2024-04-24T23:05:44.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.241+0000 7f0ef67fc700 1 -- 172.21.15.112:0/1941468405 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ef0003a30 con 0x7f0f00101e80 2024-04-24T23:05:44.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.241+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f00197590 con 0x7f0f00101e80 2024-04-24T23:05:44.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.242+0000 7f0ef67fc700 1 -- 172.21.15.112:0/1941468405 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f0ef0029070 con 0x7f0f00101e80 2024-04-24T23:05:44.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.242+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f00103360 con 0x7f0f00101e80 2024-04-24T23:05:44.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.242+0000 7f0ef67fc700 1 --2- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0eec0390f0 0x7f0eec03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:44.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.242+0000 7f0ef67fc700 1 -- 172.21.15.112:0/1941468405 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0ef001e070 con 0x7f0f00101e80 2024-04-24T23:05:44.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.242+0000 7f0f04922700 1 --2- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0eec0390f0 0x7f0eec03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:44.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.242+0000 7f0f04922700 1 --2- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0eec0390f0 0x7f0eec03b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0efc006fd0 tx=0x7f0efc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:44.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.247+0000 7f0ef67fc700 1 -- 172.21.15.112:0/1941468405 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0ef0013070 con 0x7f0f00101e80 2024-04-24T23:05:44.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:44.447+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}) -- 0x7f0f00198ac0 con 0x7f0eec0390f0 2024-04-24T23:05:45.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:45 smithi112 ceph-mon[29186]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:46.430 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:46 smithi112 ceph-mon[29186]: Deploying cephadm binary to smithi150 2024-04-24T23:05:48.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:47 smithi112 ceph-mon[29186]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:48.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.518+0000 7f0ef67fc700 1 -- 172.21.15.112:0/1941468405 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0f00198ac0 con 0x7f0eec0390f0 2024-04-24T23:05:48.521 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-04-24T23:05:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0eec0390f0 msgr2=0x7f0eec03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 --2- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0eec0390f0 0x7f0eec03b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0efc006fd0 tx=0x7f0efc006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 msgr2=0x7f0f0019d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 --2- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 0x7f0f0019d3d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ef00052a0 tx=0x7f0ef0004cc0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 shutdown_connections 2024-04-24T23:05:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 --2- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0eec0390f0 0x7f0eec03b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 --2- 172.21.15.112:0/1941468405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f00101e80 0x7f0f0019d3d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 >> 172.21.15.112:0/1941468405 conn(0x7f0f000fd750 msgr2=0x7f0f00103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.520+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 shutdown_connections 2024-04-24T23:05:48.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:48.521+0000 7f0f07387700 1 -- 172.21.15.112:0/1941468405 wait complete. 2024-04-24T23:05:49.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:05:49.329 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:49.721 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:49 smithi112 ceph-mon[29186]: Added host smithi150 2024-04-24T23:05:49.722 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:49.722 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:49 smithi112 ceph-mon[29186]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:49.722 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.736+0000 7f5b54bcf700 1 -- 172.21.15.112:0/939210333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 msgr2=0x7f5b50102250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.736+0000 7f5b54bcf700 1 --2- 172.21.15.112:0/939210333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 0x7f5b50102250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009910 tx=0x7f5b40009c20 comp rx=0 tx=0).stop 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.736+0000 7f5b54bcf700 1 -- 172.21.15.112:0/939210333 shutdown_connections 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.736+0000 7f5b54bcf700 1 --2- 172.21.15.112:0/939210333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 0x7f5b50102250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.736+0000 7f5b54bcf700 1 -- 172.21.15.112:0/939210333 >> 172.21.15.112:0/939210333 conn(0x7f5b500fd740 msgr2=0x7f5b500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.736+0000 7f5b54bcf700 1 -- 172.21.15.112:0/939210333 shutdown_connections 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.736+0000 7f5b54bcf700 1 -- 172.21.15.112:0/939210333 wait complete. 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.737+0000 7f5b54bcf700 1 Processor -- start 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.737+0000 7f5b54bcf700 1 -- start start 2024-04-24T23:05:50.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.737+0000 7f5b54bcf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 0x7f5b5019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:50.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.737+0000 7f5b54bcf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5010c380 con 0x7f5b50101e70 2024-04-24T23:05:50.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b4e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 0x7f5b5019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:50.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b4e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 0x7f5b5019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33214/0 (socket says 172.21.15.112:33214) 2024-04-24T23:05:50.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b4e59c700 1 -- 172.21.15.112:0/3167369920 learned_addr learned my addr 172.21.15.112:0/3167369920 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:50.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b4e59c700 1 -- 172.21.15.112:0/3167369920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5019d920 con 0x7f5b50101e70 2024-04-24T23:05:50.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b4e59c700 1 --2- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 0x7f5b5019d3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5b40006010 tx=0x7f5b40004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:50.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b3f7fe700 1 -- 172.21.15.112:0/3167369920 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b40026070 con 0x7f5b50101e70 2024-04-24T23:05:50.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b3f7fe700 1 -- 172.21.15.112:0/3167369920 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b400044a0 con 0x7f5b50101e70 2024-04-24T23:05:50.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5019dc10 con 0x7f5b50101e70 2024-04-24T23:05:50.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b3f7fe700 1 -- 172.21.15.112:0/3167369920 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b40003a70 con 0x7f5b50101e70 2024-04-24T23:05:50.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.738+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b501975f0 con 0x7f5b50101e70 2024-04-24T23:05:50.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.739+0000 7f5b3f7fe700 1 -- 172.21.15.112:0/3167369920 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5b40029070 con 0x7f5b50101e70 2024-04-24T23:05:50.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.739+0000 7f5b3f7fe700 1 --2- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b380390f0 0x7f5b3803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:50.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.739+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b50103350 con 0x7f5b50101e70 2024-04-24T23:05:50.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.739+0000 7f5b3f7fe700 1 -- 172.21.15.112:0/3167369920 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5b4001e070 con 0x7f5b50101e70 2024-04-24T23:05:50.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.740+0000 7f5b4dd9b700 1 --2- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b380390f0 0x7f5b3803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:50.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.740+0000 7f5b4dd9b700 1 --2- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b380390f0 0x7f5b3803b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006fd0 tx=0x7f5b44006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:50.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.744+0000 7f5b3f7fe700 1 -- 172.21.15.112:0/3167369920 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5b40030020 con 0x7f5b50101e70 2024-04-24T23:05:50.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:50.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.946+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b50198ad0 con 0x7f5b380390f0 2024-04-24T23:05:50.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.947+0000 7f5b3f7fe700 1 -- 172.21.15.112:0/3167369920 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f5b50198ad0 con 0x7f5b380390f0 2024-04-24T23:05:50.949 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:50.949 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-04-24T23:05:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b380390f0 msgr2=0x7f5b3803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 --2- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b380390f0 0x7f5b3803b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006fd0 tx=0x7f5b44006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 msgr2=0x7f5b5019d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 --2- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 0x7f5b5019d3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5b40006010 tx=0x7f5b40004cc0 comp rx=0 tx=0).stop 2024-04-24T23:05:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 shutdown_connections 2024-04-24T23:05:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 --2- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b380390f0 0x7f5b3803b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 --2- 172.21.15.112:0/3167369920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b50101e70 0x7f5b5019d3e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:50.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 >> 172.21.15.112:0/3167369920 conn(0x7f5b500fd740 msgr2=0x7f5b50103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:50.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 shutdown_connections 2024-04-24T23:05:50.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:50.949+0000 7f5b54bcf700 1 -- 172.21.15.112:0/3167369920 wait complete. 2024-04-24T23:05:51.508 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi187 2024-04-24T23:05:51.509 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:05:51.509 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:05:51.531 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:05:51.532 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:05:51.591 INFO:tasks.cephadm:Adding host smithi187 to orchestrator... 2024-04-24T23:05:51.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch host add smithi187 2024-04-24T23:05:51.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:51 smithi112 ceph-mon[29186]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:51.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:51 smithi112 ceph-mon[29186]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:05:51.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:51 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:51.825 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.273+0000 7f47dd917700 1 -- 172.21.15.112:0/1327494333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 msgr2=0x7f47d8102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.273+0000 7f47dd917700 1 --2- 172.21.15.112:0/1327494333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 0x7f47d8102260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009910 tx=0x7f47c8009c20 comp rx=0 tx=0).stop 2024-04-24T23:05:53.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.274+0000 7f47dd917700 1 -- 172.21.15.112:0/1327494333 shutdown_connections 2024-04-24T23:05:53.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.274+0000 7f47dd917700 1 --2- 172.21.15.112:0/1327494333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 0x7f47d8102260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:53.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.274+0000 7f47dd917700 1 -- 172.21.15.112:0/1327494333 >> 172.21.15.112:0/1327494333 conn(0x7f47d80fd750 msgr2=0x7f47d80ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:53.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.274+0000 7f47dd917700 1 -- 172.21.15.112:0/1327494333 shutdown_connections 2024-04-24T23:05:53.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.274+0000 7f47dd917700 1 -- 172.21.15.112:0/1327494333 wait complete. 2024-04-24T23:05:53.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.274+0000 7f47dd917700 1 Processor -- start 2024-04-24T23:05:53.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.274+0000 7f47dd917700 1 -- start start 2024-04-24T23:05:53.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47dd917700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 0x7f47d819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:53.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47dd917700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47d810c390 con 0x7f47d8101e80 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47d6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 0x7f47d819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47d6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 0x7f47d819d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33226/0 (socket says 172.21.15.112:33226) 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47d6ffd700 1 -- 172.21.15.112:0/3021908403 learned_addr learned my addr 172.21.15.112:0/3021908403 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47d6ffd700 1 -- 172.21.15.112:0/3021908403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d819d910 con 0x7f47d8101e80 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47d6ffd700 1 --2- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 0x7f47d819d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f47c80052a0 tx=0x7f47c8004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47dc915700 1 -- 172.21.15.112:0/3021908403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47c8026070 con 0x7f47d8101e80 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d819dc00 con 0x7f47d8101e80 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47dc915700 1 -- 172.21.15.112:0/3021908403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f47c800bbf0 con 0x7f47d8101e80 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47dc915700 1 -- 172.21.15.112:0/3021908403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47c8003a30 con 0x7f47d8101e80 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.275+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d8197590 con 0x7f47d8101e80 2024-04-24T23:05:53.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.276+0000 7f47dc915700 1 -- 172.21.15.112:0/3021908403 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f47c8029070 con 0x7f47d8101e80 2024-04-24T23:05:53.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.276+0000 7f47dc915700 1 --2- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47c0039150 0x7f47c003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:53.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.276+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d8103360 con 0x7f47d8101e80 2024-04-24T23:05:53.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.276+0000 7f47dc915700 1 -- 172.21.15.112:0/3021908403 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f47c801e070 con 0x7f47d8101e80 2024-04-24T23:05:53.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.276+0000 7f47d67fc700 1 --2- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47c0039150 0x7f47c003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:53.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.277+0000 7f47d67fc700 1 --2- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47c0039150 0x7f47c003b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006fd0 tx=0x7f47cc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:53.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.281+0000 7f47dc915700 1 -- 172.21.15.112:0/3021908403 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f47c8013070 con 0x7f47d8101e80 2024-04-24T23:05:53.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:53.481+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}) -- 0x7f47d8198ac0 con 0x7f47c0039150 2024-04-24T23:05:54.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:53 smithi112 ceph-mon[29186]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:55.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:54 smithi112 ceph-mon[29186]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:56.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:55 smithi112 ceph-mon[29186]: Deploying cephadm binary to smithi187 2024-04-24T23:05:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:55 smithi112 ceph-mon[29186]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-24T23:05:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:55 smithi112 ceph-mon[29186]: Updating smithi150:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:05:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:55 smithi112 ceph-mon[29186]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:57.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.598+0000 7f47dc915700 1 -- 172.21.15.112:0/3021908403 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f47d8198ac0 con 0x7f47c0039150 2024-04-24T23:05:57.600 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi187' with addr '172.21.15.187' 2024-04-24T23:05:57.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47c0039150 msgr2=0x7f47c003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:57.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 --2- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47c0039150 0x7f47c003b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f47cc006fd0 tx=0x7f47cc006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:57.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 msgr2=0x7f47d819d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:57.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 --2- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 0x7f47d819d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f47c80052a0 tx=0x7f47c8004cc0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 shutdown_connections 2024-04-24T23:05:57.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 --2- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47c0039150 0x7f47c003b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 --2- 172.21.15.112:0/3021908403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d8101e80 0x7f47d819d3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:57.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 >> 172.21.15.112:0/3021908403 conn(0x7f47d80fd750 msgr2=0x7f47d8103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:57.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 shutdown_connections 2024-04-24T23:05:57.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:57.600+0000 7f47dd917700 1 -- 172.21.15.112:0/3021908403 wait complete. 2024-04-24T23:05:57.900 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:57 smithi112 ceph-mon[29186]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:57.900 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:57.900 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:57.900 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:57.900 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:57.900 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:58.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:05:58.425 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:05:58.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:58 smithi112 ceph-mon[29186]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:05:58.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:58 smithi112 ceph-mon[29186]: Updating smithi150:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:05:58.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:58 smithi112 ceph-mon[29186]: Added host smithi187 2024-04-24T23:05:58.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:58.855 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:05:59.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.744+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3665473256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b4100da0 msgr2=0x7f61b4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.744+0000 7f61bc1d8700 1 --2- 172.21.15.112:0/3665473256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b4100da0 0x7f61b4101180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009910 tx=0x7f61b0009c20 comp rx=0 tx=0).stop 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.744+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3665473256 shutdown_connections 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.744+0000 7f61bc1d8700 1 --2- 172.21.15.112:0/3665473256 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b4100da0 0x7f61b4101180 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.744+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3665473256 >> 172.21.15.112:0/3665473256 conn(0x7f61b40fc670 msgr2=0x7f61b40fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.744+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3665473256 shutdown_connections 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.744+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3665473256 wait complete. 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61bc1d8700 1 Processor -- start 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61bc1d8700 1 -- start start 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61bc1d8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b419f880 0x7f61b419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61bc1d8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61b410b2b0 con 0x7f61b419f880 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61b9f74700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b419f880 0x7f61b419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61b9f74700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b419f880 0x7f61b419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33242/0 (socket says 172.21.15.112:33242) 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61b9f74700 1 -- 172.21.15.112:0/3381720299 learned_addr learned my addr 172.21.15.112:0/3381720299 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:05:59.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61b9f74700 1 -- 172.21.15.112:0/3381720299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b41a01a0 con 0x7f61b419f880 2024-04-24T23:05:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.745+0000 7f61b9f74700 1 --2- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b419f880 0x7f61b419fc60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61b000b5c0 tx=0x7f61b0004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.746+0000 7f61aaffd700 1 -- 172.21.15.112:0/3381720299 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0027070 con 0x7f61b419f880 2024-04-24T23:05:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.746+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b41a0490 con 0x7f61b419f880 2024-04-24T23:05:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.746+0000 7f61aaffd700 1 -- 172.21.15.112:0/3381720299 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f61b000be60 con 0x7f61b419f880 2024-04-24T23:05:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.746+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b4199c70 con 0x7f61b419f880 2024-04-24T23:05:59.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.746+0000 7f61aaffd700 1 -- 172.21.15.112:0/3381720299 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0022e10 con 0x7f61b419f880 2024-04-24T23:05:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.746+0000 7f61aaffd700 1 -- 172.21.15.112:0/3381720299 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f61b002a070 con 0x7f61b419f880 2024-04-24T23:05:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.747+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b4102280 con 0x7f61b419f880 2024-04-24T23:05:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.747+0000 7f61aaffd700 1 --2- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61a0039050 0x7f61a003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.747+0000 7f61aaffd700 1 -- 172.21.15.112:0/3381720299 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f61b001e030 con 0x7f61b419f880 2024-04-24T23:05:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.747+0000 7f61b9773700 1 --2- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61a0039050 0x7f61a003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:59.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.747+0000 7f61b9773700 1 --2- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61a0039050 0x7f61a003b510 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f61a4006fd0 tx=0x7f61a4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:59.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.752+0000 7f61aaffd700 1 -- 172.21.15.112:0/3381720299 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f61b41a0490 con 0x7f61b419f880 2024-04-24T23:05:59.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.954+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61b406a600 con 0x7f61a0039050 2024-04-24T23:05:59.956 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:05:59 smithi112 ceph-mon[29186]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:59.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.955+0000 7f61aaffd700 1 -- 172.21.15.112:0/3381720299 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f61b406a600 con 0x7f61a0039050 2024-04-24T23:05:59.957 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:05:59.957 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}, {"addr": "172.21.15.187", "hostname": "smithi187", "labels": [], "status": ""}] 2024-04-24T23:05:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61a0039050 msgr2=0x7f61a003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 --2- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61a0039050 0x7f61a003b510 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f61a4006fd0 tx=0x7f61a4006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b419f880 msgr2=0x7f61b419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 --2- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b419f880 0x7f61b419fc60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61b000b5c0 tx=0x7f61b0004cc0 comp rx=0 tx=0).stop 2024-04-24T23:05:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 shutdown_connections 2024-04-24T23:05:59.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 --2- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61a0039050 0x7f61a003b510 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 --2- 172.21.15.112:0/3381720299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61b419f880 0x7f61b419fc60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 >> 172.21.15.112:0/3381720299 conn(0x7f61b40fc670 msgr2=0x7f61b4069ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 shutdown_connections 2024-04-24T23:05:59.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:05:59.958+0000 7f61bc1d8700 1 -- 172.21.15.112:0/3381720299 wait complete. 2024-04-24T23:06:00.552 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T23:06:00.552 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T23:06:00.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:00 smithi112 ceph-mon[29186]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:06:00.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:00 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:00.787 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:06:02.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:01 smithi112 ceph-mon[29186]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:02.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.307+0000 7f5a93485700 1 -- 172.21.15.112:0/688332405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 msgr2=0x7f5a8c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:02.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.307+0000 7f5a93485700 1 --2- 172.21.15.112:0/688332405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 0x7f5a8c102260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009910 tx=0x7f5a88009c20 comp rx=0 tx=0).stop 2024-04-24T23:06:02.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.307+0000 7f5a93485700 1 -- 172.21.15.112:0/688332405 shutdown_connections 2024-04-24T23:06:02.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.307+0000 7f5a93485700 1 --2- 172.21.15.112:0/688332405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 0x7f5a8c102260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:02.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.307+0000 7f5a93485700 1 -- 172.21.15.112:0/688332405 >> 172.21.15.112:0/688332405 conn(0x7f5a8c0fd750 msgr2=0x7f5a8c0ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:02.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.307+0000 7f5a93485700 1 -- 172.21.15.112:0/688332405 shutdown_connections 2024-04-24T23:06:02.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.307+0000 7f5a93485700 1 -- 172.21.15.112:0/688332405 wait complete. 2024-04-24T23:06:02.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a93485700 1 Processor -- start 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a93485700 1 -- start start 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a93485700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 0x7f5a8c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a93485700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8c10c390 con 0x7f5a8c101e80 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a91221700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 0x7f5a8c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a91221700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 0x7f5a8c19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47490/0 (socket says 172.21.15.112:47490) 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a91221700 1 -- 172.21.15.112:0/2364924698 learned_addr learned my addr 172.21.15.112:0/2364924698 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a91221700 1 -- 172.21.15.112:0/2364924698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8c19d910 con 0x7f5a8c101e80 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a91221700 1 --2- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 0x7f5a8c19d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5a880052a0 tx=0x7f5a88004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a827fc700 1 -- 172.21.15.112:0/2364924698 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88026070 con 0x7f5a8c101e80 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a827fc700 1 -- 172.21.15.112:0/2364924698 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5a8800bbf0 con 0x7f5a8c101e80 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.308+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8c19dc00 con 0x7f5a8c101e80 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.309+0000 7f5a827fc700 1 -- 172.21.15.112:0/2364924698 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a88003a30 con 0x7f5a8c101e80 2024-04-24T23:06:02.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.309+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8c197590 con 0x7f5a8c101e80 2024-04-24T23:06:02.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.309+0000 7f5a827fc700 1 -- 172.21.15.112:0/2364924698 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5a88029070 con 0x7f5a8c101e80 2024-04-24T23:06:02.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.309+0000 7f5a827fc700 1 --2- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a78039150 0x7f5a7803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:02.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.310+0000 7f5a827fc700 1 -- 172.21.15.112:0/2364924698 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5a8801e070 con 0x7f5a8c101e80 2024-04-24T23:06:02.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.310+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a8c103360 con 0x7f5a8c101e80 2024-04-24T23:06:02.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.310+0000 7f5a90a20700 1 --2- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a78039150 0x7f5a7803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:02.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.310+0000 7f5a90a20700 1 --2- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a78039150 0x7f5a7803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c006fd0 tx=0x7f5a7c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:02.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.314+0000 7f5a827fc700 1 -- 172.21.15.112:0/2364924698 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a88030020 con 0x7f5a8c101e80 2024-04-24T23:06:02.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.512+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5a8c063890 con 0x7f5a8c101e80 2024-04-24T23:06:02.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.772+0000 7f5a827fc700 1 -- 172.21.15.112:0/2364924698 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f5a88013070 con 0x7f5a8c101e80 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.774+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a78039150 msgr2=0x7f5a7803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.774+0000 7f5a93485700 1 --2- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a78039150 0x7f5a7803b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c006fd0 tx=0x7f5a7c006e40 comp rx=0 tx=0).stop 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.774+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 msgr2=0x7f5a8c19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.774+0000 7f5a93485700 1 --2- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 0x7f5a8c19d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5a880052a0 tx=0x7f5a88004cc0 comp rx=0 tx=0).stop 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.774+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 shutdown_connections 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.774+0000 7f5a93485700 1 --2- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a78039150 0x7f5a7803b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.774+0000 7f5a93485700 1 --2- 172.21.15.112:0/2364924698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a8c101e80 0x7f5a8c19d3d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.775+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 >> 172.21.15.112:0/2364924698 conn(0x7f5a8c0fd750 msgr2=0x7f5a8c103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.775+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 shutdown_connections 2024-04-24T23:06:02.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:02.775+0000 7f5a93485700 1 -- 172.21.15.112:0/2364924698 wait complete. 2024-04-24T23:06:02.777 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-04-24T23:06:03.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:02 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2364924698' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:06:03.332 INFO:tasks.cephadm:Adding mon.a on smithi112 2024-04-24T23:06:03.332 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-04-24T23:06:03.332 INFO:tasks.cephadm:Adding mon.c on smithi187 2024-04-24T23:06:03.333 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi187:172.21.15.187=c' 2024-04-24T23:06:03.564 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:06:03.619 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2364924698' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:04.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:04.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:05.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:04 smithi112 ceph-mon[29186]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-24T23:06:05.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:04 smithi112 ceph-mon[29186]: Updating smithi187:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:05.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:04 smithi112 ceph-mon[29186]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:05.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:04 smithi112 ceph-mon[29186]: Updating smithi187:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:06:05.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:04 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:05.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:04 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:05.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:04 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:05.116 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.115+0000 7f08fe0b7700 1 -- 172.21.15.187:0/837680181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 msgr2=0x7f08f81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:05.117 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.115+0000 7f08fe0b7700 1 --2- 172.21.15.187:0/837680181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 0x7f08f81031d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f08e800b0a0 tx=0x7f08e800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:06:05.117 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.115+0000 7f08fe0b7700 1 -- 172.21.15.187:0/837680181 shutdown_connections 2024-04-24T23:06:05.117 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.115+0000 7f08fe0b7700 1 --2- 172.21.15.187:0/837680181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 0x7f08f81031d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:05.117 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.115+0000 7f08fe0b7700 1 -- 172.21.15.187:0/837680181 >> 172.21.15.187:0/837680181 conn(0x7f08f80fc680 msgr2=0x7f08f80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:05.117 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.115+0000 7f08fe0b7700 1 -- 172.21.15.187:0/837680181 shutdown_connections 2024-04-24T23:06:05.117 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.115+0000 7f08fe0b7700 1 -- 172.21.15.187:0/837680181 wait complete. 2024-04-24T23:06:05.117 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.116+0000 7f08fe0b7700 1 Processor -- start 2024-04-24T23:06:05.118 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.116+0000 7f08fe0b7700 1 -- start start 2024-04-24T23:06:05.118 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.116+0000 7f08fe0b7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 0x7f08f819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.116+0000 7f08fe0b7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08f810c420 con 0x7f08f8102df0 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08f77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 0x7f08f819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08f77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 0x7f08f819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.187:35314/0 (socket says 172.21.15.187:35314) 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08f77fe700 1 -- 172.21.15.187:0/2300794120 learned_addr learned my addr 172.21.15.187:0/2300794120 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08f77fe700 1 -- 172.21.15.187:0/2300794120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f819db70 con 0x7f08f8102df0 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08f77fe700 1 --2- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 0x7f08f819d630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08e80062a0 tx=0x7f08e8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08f4ff9700 1 -- 172.21.15.187:0/2300794120 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08e8017040 con 0x7f08f8102df0 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f819de00 con 0x7f08f8102df0 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08f4ff9700 1 -- 172.21.15.187:0/2300794120 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f08e8022070 con 0x7f08f8102df0 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08f4ff9700 1 -- 172.21.15.187:0/2300794120 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08e801d450 con 0x7f08f8102df0 2024-04-24T23:06:05.119 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.117+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f8197700 con 0x7f08f8102df0 2024-04-24T23:06:05.120 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.118+0000 7f08f4ff9700 1 -- 172.21.15.187:0/2300794120 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f08e80078a0 con 0x7f08f8102df0 2024-04-24T23:06:05.120 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.118+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f806a970 con 0x7f08f8102df0 2024-04-24T23:06:05.120 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.118+0000 7f08f4ff9700 1 --2- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e00390f0 0x7f08e003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:05.120 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.118+0000 7f08f4ff9700 1 -- 172.21.15.187:0/2300794120 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f08e800f050 con 0x7f08f8102df0 2024-04-24T23:06:05.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.118+0000 7f08f6ffd700 1 --2- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e00390f0 0x7f08e003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:05.121 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.119+0000 7f08f6ffd700 1 --2- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e00390f0 0x7f08e003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006fd0 tx=0x7f08ec006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:05.124 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.124+0000 7f08f4ff9700 1 -- 172.21.15.187:0/2300794120 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08e8004280 con 0x7f08f8102df0 2024-04-24T23:06:05.331 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.330+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}) -- 0x7f08f8198c30 con 0x7f08e00390f0 2024-04-24T23:06:05.338 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.337+0000 7f08f4ff9700 1 -- 172.21.15.187:0/2300794120 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f08f8198c30 con 0x7f08e00390f0 2024-04-24T23:06:05.338 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mon update... 2024-04-24T23:06:05.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e00390f0 msgr2=0x7f08e003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:05.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 --2- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e00390f0 0x7f08e003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006fd0 tx=0x7f08ec006e40 comp rx=0 tx=0).stop 2024-04-24T23:06:05.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 msgr2=0x7f08f819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:05.340 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 --2- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 0x7f08f819d630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f08e80062a0 tx=0x7f08e8003ce0 comp rx=0 tx=0).stop 2024-04-24T23:06:05.341 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 shutdown_connections 2024-04-24T23:06:05.341 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 --2- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e00390f0 0x7f08e003b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:05.341 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 --2- 172.21.15.187:0/2300794120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08f8102df0 0x7f08f819d630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:05.341 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 >> 172.21.15.187:0/2300794120 conn(0x7f08f80fc680 msgr2=0x7f08f8103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:05.341 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 shutdown_connections 2024-04-24T23:06:05.341 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:05.339+0000 7f08fe0b7700 1 -- 172.21.15.187:0/2300794120 wait complete. 2024-04-24T23:06:05.940 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.b.service 2024-04-24T23:06:05.943 DEBUG:teuthology.orchestra.run.smithi187:mon.c> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.c.service 2024-04-24T23:06:05.945 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:06:05.945 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:06:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:05 smithi112 ceph-mon[29186]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:06.987 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:07.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:06 smithi112 ceph-mon[29186]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:06 smithi112 ceph-mon[29186]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi187:172.21.15.187=c;count:3 2024-04-24T23:06:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:06 smithi112 ceph-mon[29186]: Deploying daemon mon.c on smithi187 2024-04-24T23:06:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:07 smithi112 ceph-mon[29186]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:09.505 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 -- 172.21.15.187:0/2093923804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 msgr2=0x7f4794106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:09.505 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 --2- 172.21.15.187:0/2093923804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 0x7f4794106fd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4788009910 tx=0x7f4788009c20 comp rx=0 tx=0).stop 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 -- 172.21.15.187:0/2093923804 shutdown_connections 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 --2- 172.21.15.187:0/2093923804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 0x7f4794106fd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 -- 172.21.15.187:0/2093923804 >> 172.21.15.187:0/2093923804 conn(0x7f47940fc630 msgr2=0x7f47940fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 -- 172.21.15.187:0/2093923804 shutdown_connections 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 -- 172.21.15.187:0/2093923804 wait complete. 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 Processor -- start 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 -- start start 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 0x7f47941985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.092+0000 7f479b06a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f479410b2f0 con 0x7f4794106bf0 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.093+0000 7f4798e06700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 0x7f47941985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.093+0000 7f4798e06700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 0x7f47941985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.187:35340/0 (socket says 172.21.15.187:35340) 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.093+0000 7f4798e06700 1 -- 172.21.15.187:0/1611472637 learned_addr learned my addr 172.21.15.187:0/1611472637 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:06:09.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.093+0000 7f4798e06700 1 -- 172.21.15.187:0/1611472637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4794198ae0 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.093+0000 7f4798e06700 1 --2- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 0x7f47941985a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47880052a0 tx=0x7f4788004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.094+0000 7f4791ffb700 1 -- 172.21.15.187:0/1611472637 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4788026070 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.094+0000 7f4791ffb700 1 -- 172.21.15.187:0/1611472637 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f478800bc50 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.094+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4794198d70 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.094+0000 7f4791ffb700 1 -- 172.21.15.187:0/1611472637 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4788003a20 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.094+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47941991d0 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.095+0000 7f4791ffb700 1 -- 172.21.15.187:0/1611472637 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4788029070 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.095+0000 7f4791ffb700 1 --2- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47840390f0 0x7f478403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.095+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47941021f0 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.095+0000 7f4791ffb700 1 -- 172.21.15.187:0/1611472637 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f478801e070 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.095+0000 7f4793fff700 1 --2- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47840390f0 0x7f478403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.096+0000 7f4793fff700 1 --2- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47840390f0 0x7f478403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4780006fd0 tx=0x7f4780006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.101+0000 7f4791ffb700 1 -- 172.21.15.187:0/1611472637 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4788030020 con 0x7f4794106bf0 2024-04-24T23:06:09.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.386+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4794063890 con 0x7f4794106bf0 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.387+0000 7f4791ffb700 1 -- 172.21.15.187:0/1611472637 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f4788013070 con 0x7f4794106bf0 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47840390f0 msgr2=0x7f478403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 --2- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47840390f0 0x7f478403b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4780006fd0 tx=0x7f4780006e40 comp rx=0 tx=0).stop 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 msgr2=0x7f47941985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 --2- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 0x7f47941985a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47880052a0 tx=0x7f4788004cc0 comp rx=0 tx=0).stop 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 shutdown_connections 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 --2- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47840390f0 0x7f478403b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 --2- 172.21.15.187:0/1611472637 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4794106bf0 0x7f47941985a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 >> 172.21.15.187:0/1611472637 conn(0x7f47940fc630 msgr2=0x7f47940fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 shutdown_connections 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:09.389+0000 7f479b06a700 1 -- 172.21.15.187:0/1611472637 wait complete. 2024-04-24T23:06:09.508 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 1 2024-04-24T23:06:09.509 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:06:09.509 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":1,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:04:25.846546Z","created":"2024-04-24T23:04:25.846546Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:06:10.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:09 smithi112 ceph-mon[29186]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:09 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/1611472637' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:10.727 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 podman[33273]: 2024-04-24 23:06:10.726821668 +0000 UTC m=+0.148538718 container create 0dfefceef0c15161601ca931bea3dcde40acffa5ea3b53c5b9d2671df1c02b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-24T23:06:10.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 podman[33273]: 2024-04-24 23:06:10.904076522 +0000 UTC m=+0.325793580 container init 0dfefceef0c15161601ca931bea3dcde40acffa5ea3b53c5b9d2671df1c02b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:06:10.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 podman[33273]: 2024-04-24 23:06:10.907894387 +0000 UTC m=+0.329611439 container start 0dfefceef0c15161601ca931bea3dcde40acffa5ea3b53c5b9d2671df1c02b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-c, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pidfile_write: ignore empty --pid-file 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: load: jerasure load: lrc 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Git sha 0 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: DB SUMMARY 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: DB Session ID: AJN8WZ61T3L0UB58ON1S 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-24T23:06:10.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.env: 0x55645a244c60 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.info_log: 0x55645c571aa0 2024-04-24T23:06:10.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.statistics: (nil) 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.use_fsync: 0 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:06:10.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.db_log_dir: 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.wal_dir: 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.write_buffer_manager: 0x55645c5441e0 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:06:10.990 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.unordered_write: 0 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.row_cache: None 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.wal_filter: None 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:06:10.991 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.wal_compression: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:06:10.992 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_open_files: -1 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:06:10.993 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Compression algorithms supported: 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: kZSTD supported: 0 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:06:10.994 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.merge_operator: 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_filter: None 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:06:10.996 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55645c571ba0) 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: index_type: 0 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: data_block_index_type: 0 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: index_shortening: 1 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: checksum: 4 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: no_block_cache: 0 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_cache: 0x55645c575090 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_cache_options: 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: capacity : 536870912 2024-04-24T23:06:10.997 INFO:journalctl@ceph.mon.c.smithi187.stdout: num_shard_bits : 4 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: strict_capacity_limit : 0 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_cache_compressed: (nil) 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: persistent_cache: (nil) 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_size: 4096 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_size_deviation: 10 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_restart_interval: 16 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: index_block_restart_interval: 1 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: metadata_block_size: 4096 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: partition_filters: 0 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: use_delta_encoding: 1 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: filter_policy: bloomfilter 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: whole_key_filtering: 1 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: verify_compression: 0 2024-04-24T23:06:10.998 INFO:journalctl@ceph.mon.c.smithi187.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout: format_version: 5 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout: enable_index_compression: 1 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout: block_align: 0 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout: max_auto_readahead_size: 262144 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout: prepopulate_block_cache: 0 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression: NoCompression 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.num_levels: 7 2024-04-24T23:06:10.999 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:06:11.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:06:11.001 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:06:11.002 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:06:11.003 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.ttl: 2592000 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:06:11.004 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3a2b4916-f99b-47b2-8e47-865509ddae70 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999970937365, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-24T23:06:11.005 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999970937807, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713999970, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3a2b4916-f99b-47b2-8e47-865509ddae70", "db_session_id": "AJN8WZ61T3L0UB58ON1S", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999970937888, "job": 1, "event": "recovery_finished"} 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55645c666000 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: DB pointer 0x55645c650000 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout: ** DB Stats ** 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:06:11.006 INFO:journalctl@ceph.mon.c.smithi187.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: ** Compaction Stats [default] ** 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: ** Compaction Stats [default] ** 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2024-04-24T23:06:11.007 INFO:journalctl@ceph.mon.c.smithi187.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: Block cache BinnedLRUCache@0x55645c575090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2024-04-24T23:06:11.008 INFO:journalctl@ceph.mon.c.smithi187.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: using public_addr v2:172.21.15.187:0/0 -> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: starting mon.c rank -1 at public addrs [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] at bind addrs [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(???) e0 preinit fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).mds e1 new map 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout: e1 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout: legacy client fscid: -1 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout: 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout: No filesystems configured 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T23:06:11.009 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:11.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Deploying cephadm binary to smithi187 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Updating smithi150:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Updating smithi150:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Added host smithi187 2024-04-24T23:06:11.011 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2364924698' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2364924698' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.012 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Updating smithi187:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Updating smithi187:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.013 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi187:172.21.15.187=c;count:3 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: Deploying daemon mon.c on smithi187 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:11.014 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/1611472637' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:11.015 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 ceph-mon[33292]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-24T23:06:11.015 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 bash[33273]: 0dfefceef0c15161601ca931bea3dcde40acffa5ea3b53c5b9d2671df1c02b2a 2024-04-24T23:06:11.015 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:10 smithi187 systemd[1]: Started Ceph mon.c for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:06:11.155 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:06:11.155 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:06:11.387 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:06:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.031+0000 7f0cd80d3700 1 -- 172.21.15.187:0/421103191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 msgr2=0x7f0cbc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.031+0000 7f0cd80d3700 1 --2- 172.21.15.187:0/421103191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 0x7f0cbc005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0101c80 tx=0x7f0cc8009e80 comp rx=0 tx=0).stop 2024-04-24T23:06:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.031+0000 7f0cd80d3700 1 -- 172.21.15.187:0/421103191 shutdown_connections 2024-04-24T23:06:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.031+0000 7f0cd80d3700 1 --2- 172.21.15.187:0/421103191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 0x7f0cbc005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.031+0000 7f0cd80d3700 1 -- 172.21.15.187:0/421103191 >> 172.21.15.187:0/421103191 conn(0x7f0cd00fd840 msgr2=0x7f0cd00ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.032+0000 7f0cd80d3700 1 -- 172.21.15.187:0/421103191 shutdown_connections 2024-04-24T23:06:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.032+0000 7f0cd80d3700 1 -- 172.21.15.187:0/421103191 wait complete. 2024-04-24T23:06:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.032+0000 7f0cd80d3700 1 Processor -- start 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd80d3700 1 -- start start 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd80d3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 0x7f0cd019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd80d3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 0x7f0cd0197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd80d3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd0109f20 con 0x7f0cbc0035c0 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd80d3700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd0109da0 con 0x7f0cd019dbc0 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd566e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 0x7f0cd0197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd566e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 0x7f0cd0197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:52678/0 (socket says 172.21.15.187:52678) 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd5e6f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 0x7f0cd019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd566e700 1 -- 172.21.15.187:0/1721639734 learned_addr learned my addr 172.21.15.187:0/1721639734 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd566e700 1 -- 172.21.15.187:0/1721639734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 msgr2=0x7f0cd0197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd566e700 1 -- 172.21.15.187:0/1721639734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 msgr2=0x7f0cd0197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd566e700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 0x7f0cd0197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:06:16.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd566e700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 0x7f0cd0197750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd5e6f700 1 -- 172.21.15.187:0/1721639734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 msgr2=0x7f0cd0197750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd5e6f700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 0x7f0cd0197750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.033+0000 7f0cd5e6f700 1 -- 172.21.15.187:0/1721639734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd019e140 con 0x7f0cbc0035c0 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.034+0000 7f0cd5e6f700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 0x7f0cd019d680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80098a0 tx=0x7f0cc8018730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.034+0000 7f0cc6ffd700 1 -- 172.21.15.187:0/1721639734 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0cc801c040 con 0x7f0cbc0035c0 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.034+0000 7f0cc6ffd700 1 -- 172.21.15.187:0/1721639734 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0cc8018b20 con 0x7f0cbc0035c0 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.034+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd0197cf0 con 0x7f0cbc0035c0 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.034+0000 7f0cc6ffd700 1 -- 172.21.15.187:0/1721639734 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0cc8029990 con 0x7f0cbc0035c0 2024-04-24T23:06:16.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.034+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd0198280 con 0x7f0cbc0035c0 2024-04-24T23:06:16.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.035+0000 7f0cc6ffd700 1 -- 172.21.15.187:0/1721639734 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f0cc8031070 con 0x7f0cbc0035c0 2024-04-24T23:06:16.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.035+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd0103490 con 0x7f0cbc0035c0 2024-04-24T23:06:16.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.035+0000 7f0cc6ffd700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0cb4038f00 0x7f0cb403b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:16.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.035+0000 7f0cc6ffd700 1 -- 172.21.15.187:0/1721639734 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0cc8026070 con 0x7f0cbc0035c0 2024-04-24T23:06:16.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.035+0000 7f0cd566e700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0cb4038f00 0x7f0cb403b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:16.040 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.036+0000 7f0cd566e700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0cb4038f00 0x7f0cb403b3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009180 tx=0x7f0cc0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:16.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.043+0000 7f0cc6ffd700 1 -- 172.21.15.187:0/1721639734 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0cc8031520 con 0x7f0cbc0035c0 2024-04-24T23:06:16.322 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.320+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0cd0198590 con 0x7f0cbc0035c0 2024-04-24T23:06:16.323 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:16.323 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:16.323 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: mon.a calling monitor election 2024-04-24T23:06:16.323 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.323 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.323 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:16.323 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.323 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: mon.c calling monitor election 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: monmap epoch 2 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: last_changed 2024-04-24T23:06:10.962827+0000 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: min_mon_release 19 (squid) 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: election_strategy: 1 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:06:16.324 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: 1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.c 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: fsmap 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: mgrmap e13: x(active, since 49s) 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: overall HEALTH_OK 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:16.325 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:16.325 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.321+0000 7f0cc6ffd700 1 -- 172.21.15.187:0/1721639734 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f0cc8038020 con 0x7f0cbc0035c0 2024-04-24T23:06:16.326 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:06:16.326 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":2,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:06:10.962827Z","created":"2024-04-24T23:04:25.846546Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0cb4038f00 msgr2=0x7f0cb403b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0cb4038f00 0x7f0cb403b3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009180 tx=0x7f0cc0008040 comp rx=0 tx=0).stop 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 msgr2=0x7f0cd019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 0x7f0cd019d680 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80098a0 tx=0x7f0cc8018730 comp rx=0 tx=0).stop 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 shutdown_connections 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0cb4038f00 0x7f0cb403b3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0cbc0035c0 0x7f0cd019d680 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 --2- 172.21.15.187:0/1721639734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0cd019dbc0 0x7f0cd0197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 >> 172.21.15.187:0/1721639734 conn(0x7f0cd00fd840 msgr2=0x7f0cd0108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 shutdown_connections 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:16.323+0000 7f0cd80d3700 1 -- 172.21.15.187:0/1721639734 wait complete. 2024-04-24T23:06:16.327 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 2 2024-04-24T23:06:16.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:16.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:16.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: mon.a calling monitor election 2024-04-24T23:06:16.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: mon.c calling monitor election 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: monmap epoch 2 2024-04-24T23:06:16.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: last_changed 2024-04-24T23:06:10.962827+0000 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: min_mon_release 19 (squid) 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: election_strategy: 1 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: 1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.c 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: fsmap 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: mgrmap e13: x(active, since 49s) 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: overall HEALTH_OK 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:16.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:16.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:16.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:17 smithi187 ceph-mon[33292]: Deploying daemon mon.b on smithi150 2024-04-24T23:06:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:17 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/1721639734' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:17 smithi187 ceph-mon[33292]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:17 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:17 smithi112 ceph-mon[29186]: Deploying daemon mon.b on smithi150 2024-04-24T23:06:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:17 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/1721639734' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:17 smithi112 ceph-mon[29186]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:17 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:17.938 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:06:17.938 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:06:18.184 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.584+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/4228055726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 msgr2=0x7f3d58101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.584+0000 7f3d5f5e3700 1 --2- 172.21.15.187:0/4228055726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 0x7f3d58101260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009910 tx=0x7f3d48009c20 comp rx=0 tx=0).stop 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.584+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/4228055726 shutdown_connections 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.584+0000 7f3d5f5e3700 1 --2- 172.21.15.187:0/4228055726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 0x7f3d58101260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.584+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/4228055726 >> 172.21.15.187:0/4228055726 conn(0x7f3d580fc710 msgr2=0x7f3d580feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.585+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/4228055726 shutdown_connections 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.585+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/4228055726 wait complete. 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.585+0000 7f3d5f5e3700 1 Processor -- start 2024-04-24T23:06:19.589 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.585+0000 7f3d5f5e3700 1 -- start start 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.585+0000 7f3d5f5e3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 0x7f3d581989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.585+0000 7f3d5f5e3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 0x7f3d58199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.585+0000 7f3d5f5e3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5810b390 con 0x7f3d58100e80 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.585+0000 7f3d5f5e3700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5810b210 con 0x7f3d58198ef0 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5d37f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 0x7f3d581989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5d37f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 0x7f3d581989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.187:58212/0 (socket says 172.21.15.187:58212) 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5d37f700 1 -- 172.21.15.187:0/3894564404 learned_addr learned my addr 172.21.15.187:0/3894564404 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5cb7e700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 0x7f3d58199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.590 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5cb7e700 1 -- 172.21.15.187:0/3894564404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 msgr2=0x7f3d58199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5cb7e700 1 -- 172.21.15.187:0/3894564404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 msgr2=0x7f3d58199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5cb7e700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 0x7f3d58199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5cb7e700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 0x7f3d58199370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5d37f700 1 -- 172.21.15.187:0/3894564404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 msgr2=0x7f3d58199370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5d37f700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 0x7f3d58199370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5d37f700 1 -- 172.21.15.187:0/3894564404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d581a1a20 con 0x7f3d58100e80 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d5d37f700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 0x7f3d581989b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d48006010 tx=0x7f3d48004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.586+0000 7f3d4e7fc700 1 -- 172.21.15.187:0/3894564404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3d48004020 con 0x7f3d58100e80 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.587+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d581a1bf0 con 0x7f3d58100e80 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.587+0000 7f3d4e7fc700 1 -- 172.21.15.187:0/3894564404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3d4800b960 con 0x7f3d58100e80 2024-04-24T23:06:19.591 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.587+0000 7f3d4e7fc700 1 -- 172.21.15.187:0/3894564404 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3d48003ce0 con 0x7f3d58100e80 2024-04-24T23:06:19.592 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.587+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d581a20a0 con 0x7f3d58100e80 2024-04-24T23:06:19.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.587+0000 7f3d4e7fc700 1 -- 172.21.15.187:0/3894564404 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3d4800bb10 con 0x7f3d58100e80 2024-04-24T23:06:19.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.587+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d58102360 con 0x7f3d58100e80 2024-04-24T23:06:19.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.588+0000 7f3d4e7fc700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3d44038e90 0x7f3d4403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:19.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.588+0000 7f3d4e7fc700 1 -- 172.21.15.187:0/3894564404 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3d4801e030 con 0x7f3d58100e80 2024-04-24T23:06:19.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.588+0000 7f3d5cb7e700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3d44038e90 0x7f3d4403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:19.593 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.588+0000 7f3d5cb7e700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3d44038e90 0x7f3d4403b350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009850 tx=0x7f3d54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:19.595 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.593+0000 7f3d4e7fc700 1 -- 172.21.15.187:0/3894564404 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3d48013070 con 0x7f3d58100e80 2024-04-24T23:06:19.868 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.866+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3d581a2370 con 0x7f3d58100e80 2024-04-24T23:06:19.868 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.867+0000 7f3d4e7fc700 1 -- 172.21.15.187:0/3894564404 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f3d4800d340 con 0x7f3d58100e80 2024-04-24T23:06:19.869 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:06:19.869 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":2,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:06:10.962827Z","created":"2024-04-24T23:04:25.846546Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3d44038e90 msgr2=0x7f3d4403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3d44038e90 0x7f3d4403b350 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009850 tx=0x7f3d54008040 comp rx=0 tx=0).stop 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 msgr2=0x7f3d581989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 0x7f3d581989b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d48006010 tx=0x7f3d48004cc0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 shutdown_connections 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3d44038e90 0x7f3d4403b350 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d58100e80 0x7f3d581989b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 --2- 172.21.15.187:0/3894564404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d58198ef0 0x7f3d58199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 >> 172.21.15.187:0/3894564404 conn(0x7f3d580fc710 msgr2=0x7f3d581028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:19.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 shutdown_connections 2024-04-24T23:06:19.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:19.869+0000 7f3d5f5e3700 1 -- 172.21.15.187:0/3894564404 wait complete. 2024-04-24T23:06:19.872 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 2 2024-04-24T23:06:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:19 smithi112 ceph-mon[29186]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:20.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:19 smithi150 systemd[1]: Started Ceph mon.b for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:06:20.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:19 smithi187 ceph-mon[33292]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:21.456 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-24T23:06:21.456 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:06:21.698 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:06:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: mon.c calling monitor election 2024-04-24T23:06:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: mon.a calling monitor election 2024-04-24T23:06:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: monmap epoch 3 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: last_changed 2024-04-24T23:06:19.928179+0000 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: min_mon_release 19 (squid) 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: election_strategy: 1 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: 1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.c 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-24T23:06:25.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: fsmap 2024-04-24T23:06:25.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:25.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: mgrmap e13: x(active, since 58s) 2024-04-24T23:06:25.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: overall HEALTH_OK 2024-04-24T23:06:25.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:25.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:25.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:25.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: mon.c calling monitor election 2024-04-24T23:06:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: mon.a calling monitor election 2024-04-24T23:06:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: monmap epoch 3 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: last_changed 2024-04-24T23:06:19.928179+0000 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: min_mon_release 19 (squid) 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: election_strategy: 1 2024-04-24T23:06:25.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: 1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.c 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: fsmap 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: mgrmap e13: x(active, since 58s) 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: overall HEALTH_OK 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:25.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.064+0000 7f864dcc5700 1 -- 172.21.15.187:0/3531763978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86300035c0 msgr2=0x7f8630005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.064+0000 7f864dcc5700 1 --2- 172.21.15.187:0/3531763978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86300035c0 0x7f8630005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8648101c80 tx=0x7f863800feb0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.065+0000 7f864dcc5700 1 -- 172.21.15.187:0/3531763978 shutdown_connections 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.065+0000 7f864dcc5700 1 --2- 172.21.15.187:0/3531763978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86300035c0 0x7f8630005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.065+0000 7f864dcc5700 1 -- 172.21.15.187:0/3531763978 >> 172.21.15.187:0/3531763978 conn(0x7f86480fd840 msgr2=0x7f86480ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.065+0000 7f864dcc5700 1 -- 172.21.15.187:0/3531763978 shutdown_connections 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.065+0000 7f864dcc5700 1 -- 172.21.15.187:0/3531763978 wait complete. 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.065+0000 7f864dcc5700 1 Processor -- start 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.065+0000 7f864dcc5700 1 -- start start 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f864dcc5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86300035c0 0x7f864819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.069 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f864dcc5700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f864819dc50 0x7f8648197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f864dcc5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8648197eb0 0x7f8648198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f864dcc5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8648109ed0 con 0x7f8648197eb0 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f864dcc5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8648109d50 con 0x7f86300035c0 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f864dcc5700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f864810a050 con 0x7f864819dc50 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f864819dc50 0x7f8648197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f864819dc50 0x7f8648197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:57398/0 (socket says 172.21.15.187:57398) 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 -- 172.21.15.187:0/2532790344 learned_addr learned my addr 172.21.15.187:0/2532790344 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f86477fe700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86300035c0 0x7f864819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8647fff700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8648197eb0 0x7f8648198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 -- 172.21.15.187:0/2532790344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86300035c0 msgr2=0x7f864819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86300035c0 0x7f864819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 -- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8648197eb0 msgr2=0x7f8648198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8648197eb0 0x7f8648198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 -- 172.21.15.187:0/2532790344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86481a45e0 con 0x7f864819dc50 2024-04-24T23:06:26.070 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8646ffd700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f864819dc50 0x7f8648197970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f863c00b780 tx=0x7f863c00bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8644ff9700 1 -- 172.21.15.187:0/2532790344 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c004230 con 0x7f864819dc50 2024-04-24T23:06:26.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8644ff9700 1 -- 172.21.15.187:0/2532790344 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f863c0043d0 con 0x7f864819dc50 2024-04-24T23:06:26.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f8644ff9700 1 -- 172.21.15.187:0/2532790344 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c011d00 con 0x7f864819dc50 2024-04-24T23:06:26.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.066+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481a4810 con 0x7f864819dc50 2024-04-24T23:06:26.071 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.067+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481a4d00 con 0x7f864819dc50 2024-04-24T23:06:26.072 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.068+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8648063890 con 0x7f864819dc50 2024-04-24T23:06:26.072 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.068+0000 7f8644ff9700 1 -- 172.21.15.187:0/2532790344 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f863c003d30 con 0x7f864819dc50 2024-04-24T23:06:26.072 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.068+0000 7f8644ff9700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8634039580 0x7f863403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:26.072 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.068+0000 7f8644ff9700 1 -- 172.21.15.187:0/2532790344 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f863c04be30 con 0x7f864819dc50 2024-04-24T23:06:26.072 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.068+0000 7f86477fe700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8634039580 0x7f863403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:26.072 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.069+0000 7f86477fe700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8634039580 0x7f863403ba40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f863800f8d0 tx=0x7f8638005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:26.075 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.073+0000 7f8644ff9700 1 -- 172.21.15.187:0/2532790344 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f863c01f200 con 0x7f864819dc50 2024-04-24T23:06:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:25 smithi187 ceph-mon[33292]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:25 smithi112 ceph-mon[29186]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:26.366 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.364+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f864802dcf0 con 0x7f864819dc50 2024-04-24T23:06:26.366 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.364+0000 7f8644ff9700 1 -- 172.21.15.187:0/2532790344 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f863c018be0 con 0x7f864819dc50 2024-04-24T23:06:26.367 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-24T23:06:26.367 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":3,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:06:19.928179Z","created":"2024-04-24T23:04:25.846546Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:3300","nonce":0},{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:06:26.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8634039580 msgr2=0x7f863403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8634039580 0x7f863403ba40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f863800f8d0 tx=0x7f8638005a90 comp rx=0 tx=0).stop 2024-04-24T23:06:26.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f864819dc50 msgr2=0x7f8648197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:26.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f864819dc50 0x7f8648197970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f863c00b780 tx=0x7f863c00bb40 comp rx=0 tx=0).stop 2024-04-24T23:06:26.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 shutdown_connections 2024-04-24T23:06:26.370 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8634039580 0x7f863403ba40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86300035c0 0x7f864819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8648197eb0 0x7f8648198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 --2- 172.21.15.187:0/2532790344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f864819dc50 0x7f8648197970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:26.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 >> 172.21.15.187:0/2532790344 conn(0x7f86480fd840 msgr2=0x7f8648100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:26.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 shutdown_connections 2024-04-24T23:06:26.371 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:26.368+0000 7f864dcc5700 1 -- 172.21.15.187:0/2532790344 wait complete. 2024-04-24T23:06:26.371 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 3 2024-04-24T23:06:27.051 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T23:06:27.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T23:06:27.297 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: mon.c calling monitor election 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: mon.a calling monitor election 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: monmap epoch 3 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: last_changed 2024-04-24T23:06:19.928179+0000 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: min_mon_release 19 (squid) 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: election_strategy: 1 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: 1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.c 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: fsmap 2024-04-24T23:06:27.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: mgrmap e13: x(active, since 58s) 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: overall HEALTH_OK 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:27.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:28.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: mon.b calling monitor election 2024-04-24T23:06:28.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-24T23:06:28.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-24T23:06:28.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-24T23:06:28.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: Updating smithi150:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: Updating smithi112:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: Updating smithi187:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: mon.b calling monitor election 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: mon.c calling monitor election 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: mon.a calling monitor election 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: monmap epoch 3 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: last_changed 2024-04-24T23:06:19.928179+0000 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: min_mon_release 19 (squid) 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: election_strategy: 1 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: 1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.c 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-24T23:06:28.031 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: fsmap 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: mgrmap e13: x(active, since 60s) 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: overall HEALTH_OK 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.032 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:28.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:28.033 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:28.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: mon.b calling monitor election 2024-04-24T23:06:28.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-24T23:06:28.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-24T23:06:28.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-24T23:06:28.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: Updating smithi150:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: Updating smithi112:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: Updating smithi187:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: mon.b calling monitor election 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: mon.c calling monitor election 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: mon.a calling monitor election 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: monmap epoch 3 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: last_changed 2024-04-24T23:06:19.928179+0000 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: min_mon_release 19 (squid) 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: election_strategy: 1 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: 1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.c 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-24T23:06:28.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: fsmap 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: mgrmap e13: x(active, since 60s) 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: overall HEALTH_OK 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:28.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:28.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: mon.b calling monitor election 2024-04-24T23:06:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-24T23:06:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: Updating smithi150:/etc/ceph/ceph.conf 2024-04-24T23:06:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-24T23:06:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: Updating smithi150:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: Updating smithi112:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: Updating smithi187:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: mon.b calling monitor election 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: mon.c calling monitor election 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: mon.a calling monitor election 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: monmap epoch 3 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: last_changed 2024-04-24T23:06:19.928179+0000 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: created 2024-04-24T23:04:25.846546+0000 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: min_mon_release 19 (squid) 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: election_strategy: 1 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-24T23:06:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: 1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.c 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: 2: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: fsmap 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: mgrmap e13: x(active, since 60s) 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: overall HEALTH_OK 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.239 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.239 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:28.239 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:28.239 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:28.239 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:28.903 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:28 smithi112 ceph-mon[29186]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:06:28.903 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:28 smithi112 ceph-mon[29186]: Reconfiguring daemon mon.a on smithi112 2024-04-24T23:06:28.904 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:29.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:28 smithi150 ceph-mon[32247]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:06:29.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:28 smithi150 ceph-mon[32247]: Reconfiguring daemon mon.a on smithi112 2024-04-24T23:06:29.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:29.198 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:06:28 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:06:28.927+0000 7f636b5ff700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T23:06:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:28 smithi187 ceph-mon[33292]: Reconfiguring mon.a (unknown last config time)... 2024-04-24T23:06:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:28 smithi187 ceph-mon[33292]: Reconfiguring daemon mon.a on smithi112 2024-04-24T23:06:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:06:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.766+0000 7ffa565ba700 1 -- 172.21.15.112:0/2258361579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa50105af0 msgr2=0x7ffa50105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.766+0000 7ffa565ba700 1 --2- 172.21.15.112:0/2258361579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa50105af0 0x7ffa50105ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009910 tx=0x7ffa40009c20 comp rx=0 tx=0).stop 2024-04-24T23:06:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.766+0000 7ffa565ba700 1 -- 172.21.15.112:0/2258361579 shutdown_connections 2024-04-24T23:06:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.766+0000 7ffa565ba700 1 --2- 172.21.15.112:0/2258361579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa50105af0 0x7ffa50105ed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.766+0000 7ffa565ba700 1 -- 172.21.15.112:0/2258361579 >> 172.21.15.112:0/2258361579 conn(0x7ffa50077220 msgr2=0x7ffa50077630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.766+0000 7ffa565ba700 1 -- 172.21.15.112:0/2258361579 shutdown_connections 2024-04-24T23:06:29.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.766+0000 7ffa565ba700 1 -- 172.21.15.112:0/2258361579 wait complete. 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa565ba700 1 Processor -- start 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa565ba700 1 -- start start 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa565ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa50105af0 0x7ffa50199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa565ba700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5019a2a0 0x7ffa5019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa565ba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa5019eb70 0x7ffa5019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa565ba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010b3b0 con 0x7ffa5019eb70 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa565ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010b230 con 0x7ffa50105af0 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa565ba700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010b530 con 0x7ffa5019a2a0 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa54b57700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa5019eb70 0x7ffa5019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa54b57700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa5019eb70 0x7ffa5019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47632/0 (socket says 172.21.15.112:47632) 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa54b57700 1 -- 172.21.15.112:0/2575045834 learned_addr learned my addr 172.21.15.112:0/2575045834 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa4ffff700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa50105af0 0x7ffa50199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.767+0000 7ffa4f7fe700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5019a2a0 0x7ffa5019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa54b57700 1 -- 172.21.15.112:0/2575045834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5019a2a0 msgr2=0x7ffa5019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa54b57700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5019a2a0 0x7ffa5019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa54b57700 1 -- 172.21.15.112:0/2575045834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa50105af0 msgr2=0x7ffa50199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa54b57700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa50105af0 0x7ffa50199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa54b57700 1 -- 172.21.15.112:0/2575045834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa501a68e0 con 0x7ffa5019eb70 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa54b57700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa5019eb70 0x7ffa5019f020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800ba50 tx=0x7ffa4800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa4d7fa700 1 -- 172.21.15.112:0/2575045834 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48019070 con 0x7ffa5019eb70 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa4d7fa700 1 -- 172.21.15.112:0/2575045834 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffa4800ce30 con 0x7ffa5019eb70 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa501a6b10 con 0x7ffa5019eb70 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa4d7fa700 1 -- 172.21.15.112:0/2575045834 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa480075b0 con 0x7ffa5019eb70 2024-04-24T23:06:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.768+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501a7000 con 0x7ffa5019eb70 2024-04-24T23:06:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.769+0000 7ffa4d7fa700 1 -- 172.21.15.112:0/2575045834 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ffa4801c070 con 0x7ffa5019eb70 2024-04-24T23:06:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.769+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa50063890 con 0x7ffa5019eb70 2024-04-24T23:06:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.769+0000 7ffa4d7fa700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffa38039580 0x7ffa3803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.769+0000 7ffa4d7fa700 1 -- 172.21.15.112:0/2575045834 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffa4804b9f0 con 0x7ffa5019eb70 2024-04-24T23:06:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.769+0000 7ffa4ffff700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffa38039580 0x7ffa3803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:29.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.770+0000 7ffa4ffff700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffa38039580 0x7ffa3803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffa40006010 tx=0x7ffa400058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.774+0000 7ffa4d7fa700 1 -- 172.21.15.112:0/2575045834 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa4801cbf0 con 0x7ffa5019eb70 2024-04-24T23:06:29.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.975+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ffa501a7230 con 0x7ffa5019eb70 2024-04-24T23:06:29.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:29 smithi112 ceph-mon[29186]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:29.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:29 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:29.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:29 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:29.977 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:29 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:29.978 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:29 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:29.978 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:29 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:29.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.975+0000 7ffa4d7fa700 1 -- 172.21.15.112:0/2575045834 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7ffa4801e620 con 0x7ffa5019eb70 2024-04-24T23:06:29.978 INFO:teuthology.orchestra.run.smithi112.stdout:# minimal ceph.conf for e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:29.978 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-24T23:06:29.978 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:29.978 INFO:teuthology.orchestra.run.smithi112.stdout: mon_host = [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] 2024-04-24T23:06:29.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.978+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffa38039580 msgr2=0x7ffa3803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.978+0000 7ffa565ba700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffa38039580 0x7ffa3803ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffa40006010 tx=0x7ffa400058e0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.978+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa5019eb70 msgr2=0x7ffa5019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:29.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.978+0000 7ffa565ba700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa5019eb70 0x7ffa5019f020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800ba50 tx=0x7ffa4800bd60 comp rx=0 tx=0).stop 2024-04-24T23:06:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.979+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 shutdown_connections 2024-04-24T23:06:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.979+0000 7ffa565ba700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffa38039580 0x7ffa3803ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.979+0000 7ffa565ba700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa50105af0 0x7ffa50199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.979+0000 7ffa565ba700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa5019eb70 0x7ffa5019f020 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.979+0000 7ffa565ba700 1 --2- 172.21.15.112:0/2575045834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa5019a2a0 0x7ffa5019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.979+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 >> 172.21.15.112:0/2575045834 conn(0x7ffa50077220 msgr2=0x7ffa50104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.979+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 shutdown_connections 2024-04-24T23:06:29.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:29.979+0000 7ffa565ba700 1 -- 172.21.15.112:0/2575045834 wait complete. 2024-04-24T23:06:30.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:29 smithi150 ceph-mon[32247]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:29 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:29 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:29 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:29 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:29 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:29 smithi187 ceph-mon[33292]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:29 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:29 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:29 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:29 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:29 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:30.555 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T23:06:30.556 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:06:30.556 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:06:30.582 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:06:30.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:30.647 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:06:30.647 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:06:30.673 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:06:30.673 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:30.738 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:06:30.738 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:06:30.769 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:06:30.769 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:30.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:30 smithi112 ceph-mon[29186]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:06:30.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:30 smithi112 ceph-mon[29186]: Reconfiguring daemon mon.b on smithi150 2024-04-24T23:06:30.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:30 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2575045834' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:30.836 INFO:tasks.cephadm:Adding mgr.x on smithi112 2024-04-24T23:06:30.837 INFO:tasks.cephadm:Adding mgr.y on smithi150 2024-04-24T23:06:30.837 INFO:tasks.cephadm:Adding mgr.z on smithi187 2024-04-24T23:06:30.837 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi112=x;smithi150=y;smithi187=z' 2024-04-24T23:06:31.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:30 smithi150 ceph-mon[32247]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:06:31.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:30 smithi150 ceph-mon[32247]: Reconfiguring daemon mon.b on smithi150 2024-04-24T23:06:31.071 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:30 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2575045834' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:31.111 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:06:31.233 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:30 smithi187 ceph-mon[33292]: Reconfiguring mon.b (monmap changed)... 2024-04-24T23:06:31.233 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:30 smithi187 ceph-mon[33292]: Reconfiguring daemon mon.b on smithi150 2024-04-24T23:06:31.233 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:30 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2575045834' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:31.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:31 smithi187 ceph-mon[33292]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:31.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:31.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:31.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:31.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:31.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:31 smithi112 ceph-mon[29186]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:32.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:31 smithi150 ceph-mon[32247]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 -- 172.21.15.187:0/585624021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a2c100cd0 msgr2=0x7f3a2c1010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 --2- 172.21.15.187:0/585624021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a2c100cd0 0x7f3a2c1010f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009910 tx=0x7f3a1c009c20 comp rx=0 tx=0).stop 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 -- 172.21.15.187:0/585624021 shutdown_connections 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 --2- 172.21.15.187:0/585624021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a2c100cd0 0x7f3a2c1010f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 -- 172.21.15.187:0/585624021 >> 172.21.15.187:0/585624021 conn(0x7f3a2c0fc270 msgr2=0x7f3a2c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 -- 172.21.15.187:0/585624021 shutdown_connections 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 -- 172.21.15.187:0/585624021 wait complete. 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 Processor -- start 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.658+0000 7f3a329ab700 1 -- start start 2024-04-24T23:06:32.663 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a329ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a2c100cd0 0x7f3a2c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a329ab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a2c197c80 0x7f3a2c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a329ab700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a2c198100 0x7f3a2c19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a329ab700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c106e50 con 0x7f3a2c197c80 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a329ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c106cd0 con 0x7f3a2c100cd0 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a329ab700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c106fd0 con 0x7f3a2c198100 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a2c198100 0x7f3a2c19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a2c198100 0x7f3a2c19f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:41110/0 (socket says 172.21.15.187:41110) 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 -- 172.21.15.187:0/536842386 learned_addr learned my addr 172.21.15.187:0/536842386 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a2b7fe700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a2c197c80 0x7f3a2c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a2bfff700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a2c100cd0 0x7f3a2c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 -- 172.21.15.187:0/536842386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a2c100cd0 msgr2=0x7f3a2c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a2c100cd0 0x7f3a2c197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.664 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 -- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a2c197c80 msgr2=0x7f3a2c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a2c197c80 0x7f3a2c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 -- 172.21.15.187:0/536842386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c19fb50 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a30f48700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a2c198100 0x7f3a2c19f280 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400ba50 tx=0x7f3a2400bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a297fa700 1 -- 172.21.15.187:0/536842386 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24019070 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a297fa700 1 -- 172.21.15.187:0/536842386 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3a2400ce30 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.659+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2c19fe40 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.660+0000 7f3a297fa700 1 -- 172.21.15.187:0/536842386 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a240075b0 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.660+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2c1a0320 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.660+0000 7f3a297fa700 1 -- 172.21.15.187:0/536842386 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3a2401c070 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.660+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2c063890 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.660+0000 7f3a297fa700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3a14039580 0x7f3a1403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.661+0000 7f3a297fa700 1 -- 172.21.15.187:0/536842386 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3a2404b940 con 0x7f3a2c198100 2024-04-24T23:06:32.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.661+0000 7f3a2bfff700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3a14039580 0x7f3a1403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:32.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.661+0000 7f3a2bfff700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3a14039580 0x7f3a1403ba40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009750 tx=0x7f3a1c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:32.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.666+0000 7f3a297fa700 1 -- 172.21.15.187:0/536842386 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a24018170 con 0x7f3a2c198100 2024-04-24T23:06:32.835 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:32 smithi187 ceph-mon[33292]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:06:32.836 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:32 smithi187 ceph-mon[33292]: Reconfiguring daemon mon.c on smithi187 2024-04-24T23:06:32.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.883+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi150=y;smithi187=z", "target": ["mon-mgr", ""]}) -- 0x7f3a2c062700 con 0x7f3a14039580 2024-04-24T23:06:32.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.892+0000 7f3a297fa700 1 -- 172.21.15.187:0/536842386 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3a2c062700 con 0x7f3a14039580 2024-04-24T23:06:32.894 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mgr update... 2024-04-24T23:06:32.896 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3a14039580 msgr2=0x7f3a1403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.896 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3a14039580 0x7f3a1403ba40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009750 tx=0x7f3a1c00b540 comp rx=0 tx=0).stop 2024-04-24T23:06:32.896 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a2c198100 msgr2=0x7f3a2c19f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:32.896 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a2c198100 0x7f3a2c19f280 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400ba50 tx=0x7f3a2400bd60 comp rx=0 tx=0).stop 2024-04-24T23:06:32.896 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 shutdown_connections 2024-04-24T23:06:32.896 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3a14039580 0x7f3a1403ba40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.896 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a2c100cd0 0x7f3a2c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.897 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a2c198100 0x7f3a2c19f280 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.897 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 --2- 172.21.15.187:0/536842386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a2c197c80 0x7f3a2c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:32.897 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 >> 172.21.15.187:0/536842386 conn(0x7f3a2c0fc270 msgr2=0x7f3a2c06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:32.897 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 shutdown_connections 2024-04-24T23:06:32.897 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:06:32.894+0000 7f3a329ab700 1 -- 172.21.15.187:0/536842386 wait complete. 2024-04-24T23:06:33.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:32 smithi112 ceph-mon[29186]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:06:33.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:32 smithi112 ceph-mon[29186]: Reconfiguring daemon mon.c on smithi187 2024-04-24T23:06:33.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:32 smithi150 ceph-mon[32247]: Reconfiguring mon.c (monmap changed)... 2024-04-24T23:06:33.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:32 smithi150 ceph-mon[32247]: Reconfiguring daemon mon.c on smithi187 2024-04-24T23:06:33.645 DEBUG:teuthology.orchestra.run.smithi150:mgr.y> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.y.service 2024-04-24T23:06:33.648 DEBUG:teuthology.orchestra.run.smithi187:mgr.z> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.z.service 2024-04-24T23:06:33.650 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T23:06:33.650 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:06:33.650 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:06:33.670 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:06:33.670 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-24T23:06:33.728 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:06:33.729 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:33.729 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-24T23:06:33.729 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:33.729 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:33.729 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-24 23:05:39.149703648 +0000 2024-04-24T23:06:33.729 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-24 23:01:04.202603161 +0000 2024-04-24T23:06:33.729 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-24 23:01:04.202603161 +0000 2024-04-24T23:06:33.729 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-24 23:01:04.202603161 +0000 2024-04-24T23:06:33.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:06:33.795 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-24T23:06:33.795 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-24T23:06:33.795 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000209924 s, 2.4 MB/s 2024-04-24T23:06:33.797 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:06:33.855 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-24T23:06:33.912 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:06:33.912 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:33.912 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-24T23:06:33.913 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:33.913 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:33.913 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-24 23:05:39.183702750 +0000 2024-04-24T23:06:33.913 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-24 23:01:04.567593589 +0000 2024-04-24T23:06:33.913 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-24 23:01:04.567593589 +0000 2024-04-24T23:06:33.913 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-24 23:01:04.567593589 +0000 2024-04-24T23:06:33.913 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:06:33.977 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-24T23:06:33.977 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-24T23:06:33.978 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000181815 s, 2.8 MB/s 2024-04-24T23:06:33.979 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:06:34.037 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-24T23:06:34.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:33 smithi150 ceph-mon[32247]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:34.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:33 smithi150 ceph-mon[32247]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi150=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:34.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:33 smithi150 ceph-mon[32247]: Saving service mgr spec with placement smithi112=x;smithi150=y;smithi187=z;count:3 2024-04-24T23:06:34.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:33 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-24 23:05:39.214701931 +0000 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-24 23:01:04.893585040 +0000 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-24 23:01:04.893585040 +0000 2024-04-24T23:06:34.092 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-24 23:01:04.893585040 +0000 2024-04-24T23:06:34.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:06:34.152 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:33 smithi112 ceph-mon[29186]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:34.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:33 smithi112 ceph-mon[29186]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi150=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:34.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:33 smithi112 ceph-mon[29186]: Saving service mgr spec with placement smithi112=x;smithi150=y;smithi187=z;count:3 2024-04-24T23:06:34.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:33 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:34.155 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-24T23:06:34.155 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-24T23:06:34.155 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000193277 s, 2.6 MB/s 2024-04-24T23:06:34.156 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:06:34.212 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-24T23:06:34.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:33 smithi187 ceph-mon[33292]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:33 smithi187 ceph-mon[33292]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi150=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:33 smithi187 ceph-mon[33292]: Saving service mgr spec with placement smithi112=x;smithi150=y;smithi187=z;count:3 2024-04-24T23:06:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:33 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:34.266 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:06:34.266 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:34.266 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-24T23:06:34.266 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:34.266 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:34.266 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-24 23:05:39.248701032 +0000 2024-04-24T23:06:34.266 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-24 23:01:05.220576464 +0000 2024-04-24T23:06:34.266 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-24 23:01:05.220576464 +0000 2024-04-24T23:06:34.267 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-24 23:01:05.220576464 +0000 2024-04-24T23:06:34.267 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:06:34.328 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-24T23:06:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-24T23:06:34.329 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000223743 s, 2.3 MB/s 2024-04-24T23:06:34.330 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:06:34.386 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-04-24T23:06:34.386 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:06:34.405 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:06:34.405 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-24 23:06:19.386475950 +0000 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-24 23:01:35.497345337 +0000 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-24 23:01:35.497345337 +0000 2024-04-24T23:06:34.463 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-24 23:01:35.497345337 +0000 2024-04-24T23:06:34.464 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:06:34.529 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-24T23:06:34.529 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-24T23:06:34.529 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000256986 s, 2.0 MB/s 2024-04-24T23:06:34.530 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:06:34.587 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-04-24T23:06:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:06:34.643 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:34.643 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T23:06:34.643 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:34.643 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:34.643 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-24 23:06:19.387475931 +0000 2024-04-24T23:06:34.643 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-24 23:01:35.830338941 +0000 2024-04-24T23:06:34.644 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-24 23:01:35.830338941 +0000 2024-04-24T23:06:34.644 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-24 23:01:35.830338941 +0000 2024-04-24T23:06:34.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:06:34.706 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-24T23:06:34.706 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-24T23:06:34.706 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000220412 s, 2.3 MB/s 2024-04-24T23:06:34.708 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:06:34.764 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-04-24T23:06:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:06:34.819 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:34.819 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-24T23:06:34.820 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:34.820 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:34.820 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-24 23:06:19.387475931 +0000 2024-04-24T23:06:34.820 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-24 23:01:36.167332468 +0000 2024-04-24T23:06:34.820 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-24 23:01:36.167332468 +0000 2024-04-24T23:06:34.820 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-24 23:01:36.167332468 +0000 2024-04-24T23:06:34.820 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:06:34.881 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-24T23:06:34.881 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-24T23:06:34.882 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000224668 s, 2.3 MB/s 2024-04-24T23:06:34.883 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:06:34.939 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-04-24T23:06:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:06:34.994 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:34.994 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-24T23:06:34.994 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:34.994 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:34.994 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-04-24 23:06:19.388475912 +0000 2024-04-24T23:06:34.994 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-04-24 23:01:36.499326092 +0000 2024-04-24T23:06:34.994 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-04-24 23:01:36.499326092 +0000 2024-04-24T23:06:34.995 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-04-24 23:01:36.499326092 +0000 2024-04-24T23:06:34.995 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:06:35.056 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-04-24T23:06:35.056 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-04-24T23:06:35.056 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000177539 s, 2.9 MB/s 2024-04-24T23:06:35.057 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:06:35.113 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-24T23:06:35.113 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:06:35.129 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:06:35.129 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 659 Links: 1 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-24 23:06:10.391019436 +0000 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-24 23:01:30.184463430 +0000 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-24 23:01:30.184463430 +0000 2024-04-24T23:06:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-24 23:01:30.184463430 +0000 2024-04-24T23:06:35.186 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:06:35.247 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-24T23:06:35.247 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-24T23:06:35.247 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.0002704 s, 1.9 MB/s 2024-04-24T23:06:35.248 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:06:35.304 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-04-24T23:06:35.317 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.317 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.317 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:35.317 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:35.317 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:35.318 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.318 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:35.318 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:06:35.318 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:35.318 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:35.318 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: Deploying daemon mgr.z on smithi187 2024-04-24T23:06:35.318 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:35 smithi187 ceph-mon[33292]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:35.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:35.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: Deploying daemon mgr.z on smithi187 2024-04-24T23:06:35.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:35 smithi150 ceph-mon[32247]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-24 23:06:10.391019436 +0000 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-24 23:01:30.524458073 +0000 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-24 23:01:30.524458073 +0000 2024-04-24T23:06:35.332 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-24 23:01:30.524458073 +0000 2024-04-24T23:06:35.333 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:06:35.395 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-24T23:06:35.395 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-24T23:06:35.395 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000214346 s, 2.4 MB/s 2024-04-24T23:06:35.396 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:06:35.452 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-04-24T23:06:35.509 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:06:35.509 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:35.509 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-04-24T23:06:35.509 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:35.509 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:35.509 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-24 23:06:10.391019436 +0000 2024-04-24T23:06:35.510 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-24 23:01:30.867452668 +0000 2024-04-24T23:06:35.510 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-24 23:01:30.867452668 +0000 2024-04-24T23:06:35.510 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-24 23:01:30.867452668 +0000 2024-04-24T23:06:35.510 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:06:35.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: Deploying daemon mgr.z on smithi187 2024-04-24T23:06:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:35 smithi112 ceph-mon[29186]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:35.572 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-24T23:06:35.572 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-24T23:06:35.572 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000196206 s, 2.6 MB/s 2024-04-24T23:06:35.573 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:06:35.630 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-24 23:06:10.392019421 +0000 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-24 23:01:31.196447485 +0000 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-24 23:01:31.196447485 +0000 2024-04-24T23:06:35.684 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-24 23:01:31.196447485 +0000 2024-04-24T23:06:35.685 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:06:35.746 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-24T23:06:35.746 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-24T23:06:35.746 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000293383 s, 1.7 MB/s 2024-04-24T23:06:35.747 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:06:35.803 INFO:tasks.cephadm:Deploying osd.0 on smithi112 with /dev/vg_nvme/lv_4... 2024-04-24T23:06:35.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:06:36.045 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:06:36.892 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:36 smithi187 systemd[1]: Starting Ceph mgr.z for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:06:37.161 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:36 smithi187 podman[35283]: 2024-04-24 23:06:36.891370971 +0000 UTC m=+0.107864310 container create 1f38b294df7c69286e6e56930259385ecf0607796321cdfc1c750b908d341ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-24T23:06:37.162 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:36 smithi187 podman[35283]: 2024-04-24 23:06:36.801687048 +0000 UTC m=+0.018180386 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:06:37.162 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 podman[35283]: 2024-04-24 23:06:37.093881151 +0000 UTC m=+0.310374489 container init 1f38b294df7c69286e6e56930259385ecf0607796321cdfc1c750b908d341ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T23:06:37.162 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 podman[35283]: 2024-04-24 23:06:37.096331614 +0000 UTC m=+0.312824963 container start 1f38b294df7c69286e6e56930259385ecf0607796321cdfc1c750b908d341ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-24T23:06:37.162 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mgr[35302]: -- 172.21.15.187:0/215816512 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564bde3625a0 con 0x564bde337000 2024-04-24T23:06:37.162 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 bash[35283]: 1f38b294df7c69286e6e56930259385ecf0607796321cdfc1c750b908d341ed9 2024-04-24T23:06:37.433 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 systemd[1]: Started Ceph mgr.z for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:06:37.433 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:37.249+0000 7fba414d4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:06:37.736 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:37.431+0000 7fba414d4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:06:37.736 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:37.531+0000 7fba414d4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:06:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:06:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:06:37.927 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-24T23:06:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-24T23:06:37.927 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131246 s, 799 MB/s 2024-04-24T23:06:37.927 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-24T23:06:38.021 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:37 smithi150 ceph-mon[32247]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:38.022 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.022 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.022 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.022 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:38.022 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:06:38.022 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:38.022 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:38.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:37 smithi112 ceph-mon[29186]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:38.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:38.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:06:38.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:38.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:38.236 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:37.839+0000 7fba414d4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:06:38.236 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:37.922+0000 7fba414d4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:06:38.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mon[33292]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:38.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:38.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:38.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:06:38.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:38.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:38.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-04-24T23:06:38.734 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:06:38.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:38 smithi187 ceph-mon[33292]: Deploying daemon mgr.y on smithi150 2024-04-24T23:06:39.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:38 smithi112 ceph-mon[29186]: Deploying daemon mgr.y on smithi150 2024-04-24T23:06:39.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:38 smithi150 ceph-mon[32247]: Deploying daemon mgr.y on smithi150 2024-04-24T23:06:39.472 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:39.144+0000 7fba414d4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:06:39.472 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:39.313+0000 7fba414d4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:06:39.545 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 systemd[1]: Starting Ceph mgr.y for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:06:39.545 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 podman[33620]: 2024-04-24 23:06:39.447002397 +0000 UTC m=+0.018045847 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:06:39.736 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:39.487+0000 7fba414d4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:06:39.803 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:39 smithi150 ceph-mon[32247]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:39.803 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 podman[33620]: 2024-04-24 23:06:39.565395083 +0000 UTC m=+0.136438523 container create 17588345cc0cddd8620333cf9bcb2799c775136f58f69e797843148362945d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:06:39.803 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 podman[33620]: 2024-04-24 23:06:39.775912461 +0000 UTC m=+0.346955900 container init 17588345cc0cddd8620333cf9bcb2799c775136f58f69e797843148362945d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T23:06:39.803 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 podman[33620]: 2024-04-24 23:06:39.77845816 +0000 UTC m=+0.349501600 container start 17588345cc0cddd8620333cf9bcb2799c775136f58f69e797843148362945d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T23:06:40.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:39 smithi112 ceph-mon[29186]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:40.068 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 ceph-mgr[33638]: -- 172.21.15.150:0/1495508174 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562da8f8c5a0 con 0x562da8f60c00 2024-04-24T23:06:40.068 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 bash[33620]: 17588345cc0cddd8620333cf9bcb2799c775136f58f69e797843148362945d8b 2024-04-24T23:06:40.068 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 systemd[1]: Started Ceph mgr.y for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:06:40.068 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:39 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:39.933+0000 7f1d5104d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T23:06:40.072 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:39 smithi187 ceph-mon[33292]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:40.072 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:39.771+0000 7fba414d4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:06:40.072 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:39.863+0000 7fba414d4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:06:40.072 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:40.070+0000 7fba414d4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:06:40.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 -- 172.21.15.112:0/2319928174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 msgr2=0x7f44d810a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:40.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 --2- 172.21.15.112:0/2319928174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 0x7f44d810a890 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f44d000b0d0 tx=0x7f44d000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:06:40.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 -- 172.21.15.112:0/2319928174 shutdown_connections 2024-04-24T23:06:40.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 --2- 172.21.15.112:0/2319928174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 0x7f44d810a890 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:40.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 --2- 172.21.15.112:0/2319928174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44d81005c0 0x7f44d81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:40.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 --2- 172.21.15.112:0/2319928174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d8103640 0x7f44d8103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:40.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 -- 172.21.15.112:0/2319928174 >> 172.21.15.112:0/2319928174 conn(0x7f44d80fbf00 msgr2=0x7f44d80fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:40.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 -- 172.21.15.112:0/2319928174 shutdown_connections 2024-04-24T23:06:40.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 -- 172.21.15.112:0/2319928174 wait complete. 2024-04-24T23:06:40.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 Processor -- start 2024-04-24T23:06:40.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.187+0000 7f44de452700 1 -- start start 2024-04-24T23:06:40.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44de452700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d81005c0 0x7f44d8102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44de452700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44d8103640 0x7f44d81015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44de452700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 0x7f44d8101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44de452700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f44d810e000 con 0x7f44d8103fe0 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44de452700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f44d810de80 con 0x7f44d81005c0 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44de452700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44d810e180 con 0x7f44d8103640 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 0x7f44d8101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 0x7f44d8101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47526/0 (socket says 172.21.15.112:47526) 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 -- 172.21.15.112:0/1875600327 learned_addr learned my addr 172.21.15.112:0/1875600327 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44d77fe700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44d8103640 0x7f44d81015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44d7fff700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d81005c0 0x7f44d8102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 -- 172.21.15.112:0/1875600327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44d8103640 msgr2=0x7f44d81015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44d8103640 0x7f44d81015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 -- 172.21.15.112:0/1875600327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d81005c0 msgr2=0x7f44d8102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:40.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d81005c0 0x7f44d8102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 -- 172.21.15.112:0/1875600327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d81023e0 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44dc9ef700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 0x7f44d8101b30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44d0007dc0 tx=0x7f44d0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d001f020 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d81a25b0 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f44d0003e10 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.188+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d001b7e0 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.189+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d81a2a00 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.189+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f44d0022020 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.189+0000 7f44d57fa700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f44c0039330 0x7f44c003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.190+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f44d000f020 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.190+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d8068f20 con 0x7f44d8103fe0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.190+0000 7f44d7fff700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f44c0039330 0x7f44c003b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:40.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.190+0000 7f44d7fff700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f44c0039330 0x7f44c003b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44d806a6a0 tx=0x7f44c8007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:40.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.195+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f44d0022310 con 0x7f44d8103fe0 2024-04-24T23:06:40.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:40.399+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f44d8062670 con 0x7f44c0039330 2024-04-24T23:06:40.485 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:40.271+0000 7fba414d4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:06:40.568 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:40.126+0000 7f1d5104d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:06:40.568 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:40.235+0000 7f1d5104d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T23:06:40.873 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:40.567+0000 7f1d5104d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T23:06:40.874 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:40.667+0000 7f1d5104d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T23:06:41.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:41.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:41.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:41.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:41.236 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:40.908+0000 7fba414d4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:06:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:41.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:41.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:41.635 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:41.341+0000 7fba414d4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:06:41.635 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:41.438+0000 7fba414d4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:06:41.635 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:41.633+0000 7fba414d4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:06:42.166 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:42.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:42.151+0000 7fba414d4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:42.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:42.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:42.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.237 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:41.895+0000 7f1d5104d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T23:06:42.237 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:42.063+0000 7f1d5104d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T23:06:42.237 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:42.235+0000 7f1d5104d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T23:06:42.237 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:42.238 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:42.606 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:42.241+0000 7fba414d4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:06:42.606 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:42.474+0000 7fba414d4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:06:42.606 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:42.604+0000 7fba414d4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:06:42.818 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:42.521+0000 7f1d5104d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T23:06:42.818 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:42.612+0000 7f1d5104d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T23:06:42.903 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:42.697+0000 7fba414d4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:06:42.903 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:42.811+0000 7fba414d4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:06:42.903 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:42.900+0000 7fba414d4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:06:43.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-mon[33292]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:06:43.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:42 smithi187 ceph-mon[33292]: Reconfiguring daemon mgr.x on smithi112 2024-04-24T23:06:43.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:42 smithi112 ceph-mon[29186]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:06:43.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:42 smithi112 ceph-mon[29186]: Reconfiguring daemon mgr.x on smithi112 2024-04-24T23:06:43.322 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:42 smithi150 ceph-mon[32247]: Reconfiguring mgr.x (unknown last config time)... 2024-04-24T23:06:43.322 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:42 smithi150 ceph-mon[32247]: Reconfiguring daemon mgr.x on smithi112 2024-04-24T23:06:43.322 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:42.816+0000 7f1d5104d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T23:06:43.322 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:43.006+0000 7f1d5104d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T23:06:43.957 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:43.605+0000 7f1d5104d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T23:06:43.963 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:43.704+0000 7fba414d4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:06:43.963 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z[35298]: 2024-04-24T23:06:43.914+0000 7fba414d4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:06:43.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:43.962+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f44d0024790 con 0x7f44d8103fe0 2024-04-24T23:06:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4199649083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7d3b0e-ce0f-4b59-955f-98303b9bd863"}]: dispatch 2024-04-24T23:06:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4199649083' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c7d3b0e-ce0f-4b59-955f-98303b9bd863"}]': finished 2024-04-24T23:06:44.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:06:44.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:06:44.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/832098921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:44.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: Standby manager daemon z started 2024-04-24T23:06:44.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:06:44.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:06:44.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:06:44.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:43 smithi187 ceph-mon[33292]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:06:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4199649083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7d3b0e-ce0f-4b59-955f-98303b9bd863"}]: dispatch 2024-04-24T23:06:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4199649083' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c7d3b0e-ce0f-4b59-955f-98303b9bd863"}]': finished 2024-04-24T23:06:44.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:06:44.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:06:44.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/832098921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:44.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: Standby manager daemon z started 2024-04-24T23:06:44.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:06:44.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:06:44.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:06:44.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:43 smithi112 ceph-mon[29186]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:06:44.314 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:44.020+0000 7f1d5104d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T23:06:44.314 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:44.113+0000 7f1d5104d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T23:06:44.314 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:44.311+0000 7f1d5104d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T23:06:44.314 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:44.314 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4199649083' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c7d3b0e-ce0f-4b59-955f-98303b9bd863"}]: dispatch 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4199649083' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c7d3b0e-ce0f-4b59-955f-98303b9bd863"}]': finished 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: osdmap e5: 1 total, 0 up, 1 in 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/832098921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: Standby manager daemon z started 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-24T23:06:44.315 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:43 smithi150 ceph-mon[32247]: from='mgr.? 172.21.15.187:0/2854355668' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:06:45.164 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:44 smithi150 ceph-mon[32247]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:06:45.165 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:44 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:06:45.165 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:44.832+0000 7f1d5104d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T23:06:45.165 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:44.921+0000 7f1d5104d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T23:06:45.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:44 smithi187 ceph-mon[33292]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:06:45.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:44 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:06:45.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:44 smithi112 ceph-mon[29186]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-24T23:06:45.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:44 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-24T23:06:45.502 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:45.162+0000 7f1d5104d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T23:06:45.502 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:45.293+0000 7f1d5104d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T23:06:45.502 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:45.386+0000 7f1d5104d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T23:06:45.502 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:45.499+0000 7f1d5104d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T23:06:45.818 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:45.589+0000 7f1d5104d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T23:06:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:45 smithi187 ceph-mon[33292]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:45 smithi112 ceph-mon[29186]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:46.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:45 smithi150 ceph-mon[32247]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:46.818 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:46 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:46.410+0000 7f1d5104d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T23:06:46.818 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:06:46 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y[33634]: 2024-04-24T23:06:46.620+0000 7f1d5104d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T23:06:46.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:46.979+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f44d0011390 con 0x7f44d8103fe0 2024-04-24T23:06:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:46 smithi187 ceph-mon[33292]: Standby manager daemon y started 2024-04-24T23:06:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:46 smithi187 ceph-mon[33292]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:06:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:46 smithi187 ceph-mon[33292]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:06:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:46 smithi187 ceph-mon[33292]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:06:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:46 smithi187 ceph-mon[33292]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:06:47.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:46 smithi112 ceph-mon[29186]: Standby manager daemon y started 2024-04-24T23:06:47.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:46 smithi112 ceph-mon[29186]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:06:47.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:46 smithi112 ceph-mon[29186]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:06:47.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:46 smithi112 ceph-mon[29186]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:06:47.249 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:46 smithi112 ceph-mon[29186]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:06:47.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:46 smithi150 ceph-mon[32247]: Standby manager daemon y started 2024-04-24T23:06:47.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:46 smithi150 ceph-mon[32247]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-24T23:06:47.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:46 smithi150 ceph-mon[32247]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:06:47.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:46 smithi150 ceph-mon[32247]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-24T23:06:47.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:46 smithi150 ceph-mon[32247]: from='mgr.? 172.21.15.150:0/2420093565' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:06:48.206 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:47 smithi112 ceph-mon[29186]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:48.206 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:47 smithi112 ceph-mon[29186]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-24T23:06:48.206 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:06:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:47 smithi187 ceph-mon[33292]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:47 smithi187 ceph-mon[33292]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-24T23:06:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:06:48.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:47 smithi150 ceph-mon[32247]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:48.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:47 smithi150 ceph-mon[32247]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-24T23:06:48.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-24T23:06:49.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:49.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:49.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:49.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:49 smithi112 ceph-mon[29186]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:49.533 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.533 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.533 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:49.533 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:49.533 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:49.533 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.533 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:49 smithi187 ceph-mon[33292]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:49 smithi150 ceph-mon[32247]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:06:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:50.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:06:50.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:50.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:06:50.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:51.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:51 smithi112 ceph-mon[29186]: Deploying daemon osd.0 on smithi112 2024-04-24T23:06:51.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:51 smithi112 ceph-mon[29186]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:51.735 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:51 smithi187 ceph-mon[33292]: Deploying daemon osd.0 on smithi112 2024-04-24T23:06:51.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:51 smithi187 ceph-mon[33292]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:51.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:51 smithi150 ceph-mon[32247]: Deploying daemon osd.0 on smithi112 2024-04-24T23:06:51.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:51 smithi150 ceph-mon[32247]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:53.994 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:53 smithi112 ceph-mon[29186]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:54.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:53 smithi150 ceph-mon[32247]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:53 smithi187 ceph-mon[33292]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:56.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:55 smithi112 ceph-mon[29186]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:56.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:55 smithi150 ceph-mon[32247]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:56.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:56.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:56.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:55 smithi187 ceph-mon[33292]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:57.970 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:57 smithi112 ceph-mon[29186]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:57.970 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:57.970 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:57.970 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:57.971 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:57.971 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:57 smithi150 ceph-mon[32247]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:57 smithi187 ceph-mon[33292]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:06:59.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.182+0000 7f44d57fa700 1 -- 172.21.15.112:0/1875600327 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f44d8062670 con 0x7f44c0039330 2024-04-24T23:06:59.184 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 0 on host 'smithi112' 2024-04-24T23:06:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f44c0039330 msgr2=0x7f44c003b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f44c0039330 0x7f44c003b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f44d806a6a0 tx=0x7f44c8007400 comp rx=0 tx=0).stop 2024-04-24T23:06:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 msgr2=0x7f44d8101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 0x7f44d8101b30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44d0007dc0 tx=0x7f44d0007df0 comp rx=0 tx=0).stop 2024-04-24T23:06:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 shutdown_connections 2024-04-24T23:06:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f44c0039330 0x7f44c003b7f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d81005c0 0x7f44d8102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:59.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f44d8103fe0 0x7f44d8101b30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:59.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 --2- 172.21.15.112:0/1875600327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44d8103640 0x7f44d81015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:59.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 >> 172.21.15.112:0/1875600327 conn(0x7f44d80fbf00 msgr2=0x7f44d8106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:59.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 shutdown_connections 2024-04-24T23:06:59.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:06:59.184+0000 7f44de452700 1 -- 172.21.15.112:0/1875600327 wait complete. 2024-04-24T23:06:59.825 DEBUG:teuthology.orchestra.run.smithi112:osd.0> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T23:06:59.827 INFO:tasks.cephadm:Deploying osd.1 on smithi150 with /dev/vg_nvme/lv_4... 2024-04-24T23:06:59.827 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:07:00.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:59 smithi150 ceph-mon[32247]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:00.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:59 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:00.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:59 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:00.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:59 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:00.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:06:59 smithi150 ceph-mon[32247]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:07:00.074 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:07:00.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:59 smithi112 ceph-mon[29186]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:00.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:59 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:00.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:59 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:00.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:59 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:00.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:06:59 smithi112 ceph-mon[29186]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:07:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:59 smithi187 ceph-mon[33292]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:59 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:59 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:59 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:06:59 smithi187 ceph-mon[33292]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:07:01.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:01 smithi150 ceph-mon[32247]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:07:01.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:01 smithi150 ceph-mon[32247]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:07:01.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:01 smithi150 ceph-mon[32247]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:07:01.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:01.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:01 smithi150 ceph-mon[32247]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:01.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:01 smithi187 ceph-mon[33292]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:07:01.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:01 smithi187 ceph-mon[33292]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:07:01.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:01 smithi187 ceph-mon[33292]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:07:01.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:01.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:01 smithi187 ceph-mon[33292]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:01 smithi112 ceph-mon[29186]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:07:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:01 smithi112 ceph-mon[29186]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:07:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:01 smithi112 ceph-mon[29186]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:07:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:01 smithi112 ceph-mon[29186]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:01.941 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:07:01.941 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:07:01.941 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-24T23:07:01.941 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-24T23:07:01.942 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129679 s, 809 MB/s 2024-04-24T23:07:01.942 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-24T23:07:02.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:02 smithi112 ceph-mon[29186]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:07:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:02 smithi112 ceph-mon[29186]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:07:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:02 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:02 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:02 smithi112 ceph-mon[29186]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' 2024-04-24T23:07:02.279 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:01 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0[43928]: 2024-04-24T23:07:01.958+0000 7f76a2509700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:07:02.279 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:01 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0[43928]: 2024-04-24T23:07:01.962+0000 7f769a2f4700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:02.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:02 smithi187 ceph-mon[33292]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:07:02.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:02 smithi187 ceph-mon[33292]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:07:02.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:02 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:02.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:02 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:02.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:02 smithi187 ceph-mon[33292]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' 2024-04-24T23:07:02.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:02 smithi150 ceph-mon[32247]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:07:02.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:02 smithi150 ceph-mon[32247]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:07:02.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:02 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:02.538 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:02 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:02.539 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:02 smithi150 ceph-mon[32247]: from='osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369]' entity='osd.0' 2024-04-24T23:07:02.566 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-04-24T23:07:02.810 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:07:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:07:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:07:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi112 2024-04-24T23:07:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] boot 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:07:03.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:07:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:07:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi112 2024-04-24T23:07:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:03.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-24T23:07:03.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:03.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:03.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] boot 2024-04-24T23:07:03.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:07:03.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi112 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:03.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:03.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: osd.0 [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] boot 2024-04-24T23:07:03.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: osdmap e8: 1 total, 1 up, 1 in 2024-04-24T23:07:03.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:04.328 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 -- 172.21.15.150:0/2862967101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103e30 msgr2=0x7f6de010a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 --2- 172.21.15.150:0/2862967101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103e30 0x7f6de010a6e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800b0d0 tx=0x7f6dd800b3e0 comp rx=0 tx=0).stop 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 -- 172.21.15.150:0/2862967101 shutdown_connections 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 --2- 172.21.15.150:0/2862967101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103e30 0x7f6de010a6e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 --2- 172.21.15.150:0/2862967101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0102290 0x7f6de01026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 --2- 172.21.15.150:0/2862967101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6de0103490 0x7f6de01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 -- 172.21.15.150:0/2862967101 >> 172.21.15.150:0/2862967101 conn(0x7f6de00fd830 msgr2=0x7f6de00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 -- 172.21.15.150:0/2862967101 shutdown_connections 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.325+0000 7f6de50f7700 1 -- 172.21.15.150:0/2862967101 wait complete. 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6de50f7700 1 Processor -- start 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6de50f7700 1 -- start start 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6de50f7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0102290 0x7f6de019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6de50f7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103490 0x7f6de019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6de50f7700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6de0103e30 0x7f6de01a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:04.329 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6de50f7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010e0b0 con 0x7f6de0102290 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6de50f7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010df30 con 0x7f6de0103490 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6de50f7700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6de010e230 con 0x7f6de0103e30 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6dde59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103490 0x7f6de019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6dde59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103490 0x7f6de019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59174/0 (socket says 172.21.15.150:59174) 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.326+0000 7f6dde59c700 1 -- 172.21.15.150:0/1064395938 learned_addr learned my addr 172.21.15.150:0/1064395938 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6ddf59e700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6de0103e30 0x7f6de01a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dded9d700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0102290 0x7f6de019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dde59c700 1 -- 172.21.15.150:0/1064395938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6de0103e30 msgr2=0x7f6de01a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dde59c700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6de0103e30 0x7f6de01a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dde59c700 1 -- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0102290 msgr2=0x7f6de019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dde59c700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0102290 0x7f6de019a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:04.330 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dde59c700 1 -- 172.21.15.150:0/1064395938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de01a24d0 con 0x7f6de0103490 2024-04-24T23:07:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dde59c700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103490 0x7f6de019adb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400b530 tx=0x7f6dd400b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dc7fff700 1 -- 172.21.15.150:0/1064395938 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4015020 con 0x7f6de0103490 2024-04-24T23:07:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de01a27c0 con 0x7f6de0103490 2024-04-24T23:07:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dc7fff700 1 -- 172.21.15.150:0/1064395938 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6dd4011d30 con 0x7f6de0103490 2024-04-24T23:07:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6dc7fff700 1 -- 172.21.15.150:0/1064395938 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400f2e0 con 0x7f6de0103490 2024-04-24T23:07:04.331 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.327+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01a2d80 con 0x7f6de0103490 2024-04-24T23:07:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.328+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de0068f20 con 0x7f6de0103490 2024-04-24T23:07:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.329+0000 7f6dc7fff700 1 -- 172.21.15.150:0/1064395938 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f6dd4018020 con 0x7f6de0103490 2024-04-24T23:07:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.330+0000 7f6dc7fff700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6dc80a2b90 0x7f6dc80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.330+0000 7f6dc7fff700 1 -- 172.21.15.150:0/1064395938 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f6dd40ccda0 con 0x7f6de0103490 2024-04-24T23:07:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.330+0000 7f6dded9d700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6dc80a2b90 0x7f6dc80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:04.332 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.330+0000 7f6dded9d700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6dc80a2b90 0x7f6dc80a5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0006fd0 tx=0x7f6dd0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:04.336 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.334+0000 7f6dc7fff700 1 -- 172.21.15.150:0/1064395938 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6dd401f020 con 0x7f6de0103490 2024-04-24T23:07:04.538 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:04.537+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6de0075b80 con 0x7f6dc80a2b90 2024-04-24T23:07:05.403 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:05 smithi150 ceph-mon[32247]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:07:05.403 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:05 smithi150 ceph-mon[32247]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:05.403 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:05.403 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:05.404 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:05.404 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:05 smithi150 ceph-mon[32247]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:05 smithi187 ceph-mon[33292]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:07:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:05 smithi187 ceph-mon[33292]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:05 smithi187 ceph-mon[33292]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:05 smithi112 ceph-mon[29186]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:07:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:05 smithi112 ceph-mon[29186]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:05 smithi112 ceph-mon[29186]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:07 smithi112 ceph-mon[29186]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:07 smithi112 ceph-mon[29186]: from='client.? 172.21.15.150:0/4022276923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]: dispatch 2024-04-24T23:07:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:07 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]: dispatch 2024-04-24T23:07:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:07 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]': finished 2024-04-24T23:07:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:07 smithi112 ceph-mon[29186]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:07:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:07 smithi112 ceph-mon[29186]: from='client.? 172.21.15.150:0/4200821302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:08.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:07 smithi150 ceph-mon[32247]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:08.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:07 smithi150 ceph-mon[32247]: from='client.? 172.21.15.150:0/4022276923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]: dispatch 2024-04-24T23:07:08.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:07 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]: dispatch 2024-04-24T23:07:08.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:07 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]': finished 2024-04-24T23:07:08.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:07 smithi150 ceph-mon[32247]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:07:08.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:08.070 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:07 smithi150 ceph-mon[32247]: from='client.? 172.21.15.150:0/4200821302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:08.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:07 smithi187 ceph-mon[33292]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:08.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:07 smithi187 ceph-mon[33292]: from='client.? 172.21.15.150:0/4022276923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]: dispatch 2024-04-24T23:07:08.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:07 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]: dispatch 2024-04-24T23:07:08.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:07 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a46ec67e-696c-497b-a0f4-9a38bae79a8f"}]': finished 2024-04-24T23:07:08.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:07 smithi187 ceph-mon[33292]: osdmap e10: 2 total, 1 up, 2 in 2024-04-24T23:07:08.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:08.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:07 smithi187 ceph-mon[33292]: from='client.? 172.21.15.150:0/4200821302' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:10.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:09 smithi112 ceph-mon[29186]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:10.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:09 smithi150 ceph-mon[32247]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:09 smithi187 ceph-mon[33292]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:12.042 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:11 smithi150 ceph-mon[32247]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:12.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:11 smithi187 ceph-mon[33292]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:12.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:11 smithi112 ceph-mon[29186]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:14.057 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:13 smithi150 ceph-mon[32247]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:13 smithi187 ceph-mon[33292]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:14.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:13 smithi112 ceph-mon[29186]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:15.042 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:07:15.042 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:07:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:15.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:07:15.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:16.047 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:15 smithi150 ceph-mon[32247]: Deploying daemon osd.1 on smithi150 2024-04-24T23:07:16.047 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:15 smithi150 ceph-mon[32247]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:16.180 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:15 smithi112 ceph-mon[29186]: Deploying daemon osd.1 on smithi150 2024-04-24T23:07:16.180 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:15 smithi112 ceph-mon[29186]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:15 smithi187 ceph-mon[33292]: Deploying daemon osd.1 on smithi150 2024-04-24T23:07:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:15 smithi187 ceph-mon[33292]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:18.016 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:17 smithi150 ceph-mon[32247]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:17 smithi187 ceph-mon[33292]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:17 smithi112 ceph-mon[29186]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:20.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:19 smithi150 ceph-mon[32247]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:20.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:20.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:20.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:19 smithi187 ceph-mon[33292]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:20.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:19 smithi112 ceph-mon[29186]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:20.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:20.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:20.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:21.891 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:21 smithi150 ceph-mon[32247]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:21 smithi187 ceph-mon[33292]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:22.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:21 smithi112 ceph-mon[29186]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:23.052 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.050+0000 7f6dc7fff700 1 -- 172.21.15.150:0/1064395938 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6de0075b80 con 0x7f6dc80a2b90 2024-04-24T23:07:23.052 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 1 on host 'smithi150' 2024-04-24T23:07:23.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6dc80a2b90 msgr2=0x7f6dc80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:23.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6dc80a2b90 0x7f6dc80a5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0006fd0 tx=0x7f6dd0008040 comp rx=0 tx=0).stop 2024-04-24T23:07:23.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103490 msgr2=0x7f6de019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:23.054 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103490 0x7f6de019adb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400b530 tx=0x7f6dd400b8f0 comp rx=0 tx=0).stop 2024-04-24T23:07:23.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 shutdown_connections 2024-04-24T23:07:23.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6dc80a2b90 0x7f6dc80a5050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:23.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6de0103e30 0x7f6de01a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:23.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6de0102290 0x7f6de019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:23.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.052+0000 7f6de50f7700 1 --2- 172.21.15.150:0/1064395938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6de0103490 0x7f6de019adb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:23.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.053+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 >> 172.21.15.150:0/1064395938 conn(0x7f6de00fd830 msgr2=0x7f6de0074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:23.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.053+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 shutdown_connections 2024-04-24T23:07:23.055 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:07:23.053+0000 7f6de50f7700 1 -- 172.21.15.150:0/1064395938 wait complete. 2024-04-24T23:07:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.684 DEBUG:teuthology.orchestra.run.smithi150:osd.1> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T23:07:23.687 INFO:tasks.cephadm:Deploying osd.2 on smithi187 with /dev/vg_nvme/lv_4... 2024-04-24T23:07:23.687 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.714 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:23.927 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:07:24.321 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:24 smithi187 ceph-mon[33292]: from='osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:07:24.321 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:24 smithi187 ceph-mon[33292]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:07:24.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:24 smithi112 ceph-mon[29186]: from='osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:07:24.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:24 smithi112 ceph-mon[29186]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:07:24.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:24 smithi150 ceph-mon[32247]: from='osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:07:24.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:24 smithi150 ceph-mon[32247]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:07:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:25 smithi187 ceph-mon[33292]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:07:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:25 smithi187 ceph-mon[33292]: from='osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:07:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:25 smithi187 ceph-mon[33292]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:07:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:25 smithi187 ceph-mon[33292]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:07:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:25 smithi187 ceph-mon[33292]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:25 smithi150 ceph-mon[32247]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:07:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:25 smithi150 ceph-mon[32247]: from='osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:07:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:25 smithi150 ceph-mon[32247]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:07:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:25 smithi150 ceph-mon[32247]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:07:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:25 smithi150 ceph-mon[32247]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:25.646 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:07:25.646 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:07:25.647 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2024-04-24T23:07:25.647 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2024-04-24T23:07:25.647 INFO:teuthology.orchestra.run.smithi187.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150044 s, 699 MB/s 2024-04-24T23:07:25.647 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 2024-04-24T23:07:25.647 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2024-04-24T23:07:25.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:25 smithi112 ceph-mon[29186]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:07:25.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:25 smithi112 ceph-mon[29186]: from='osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:07:25.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:25 smithi112 ceph-mon[29186]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:07:25.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:25.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:25 smithi112 ceph-mon[29186]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:07:25.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:25 smithi112 ceph-mon[29186]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:26.258 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi187:vg_nvme/lv_4 2024-04-24T23:07:26.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:26 smithi150 ceph-mon[32247]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:07:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:26 smithi150 ceph-mon[32247]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:07:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:26 smithi150 ceph-mon[32247]: from='osd.1 ' entity='osd.1' 2024-04-24T23:07:26.318 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:26 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1[36843]: 2024-04-24T23:07:26.039+0000 7f8dd20b6700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:07:26.318 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:26 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1[36843]: 2024-04-24T23:07:26.043+0000 7f8dc9ea1700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:26.321 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:26 smithi187 ceph-mon[33292]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:07:26.321 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:26 smithi187 ceph-mon[33292]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:07:26.321 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:26.321 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:26.321 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:26 smithi187 ceph-mon[33292]: from='osd.1 ' entity='osd.1' 2024-04-24T23:07:26.536 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:07:26.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:26 smithi112 ceph-mon[29186]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:07:26.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:26 smithi112 ceph-mon[29186]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:07:26.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:26.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:26.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:26 smithi112 ceph-mon[29186]: from='osd.1 ' entity='osd.1' 2024-04-24T23:07:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:07:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:07:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi150 2024-04-24T23:07:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-24T23:07:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] boot 2024-04-24T23:07:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:07:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi150 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:27.658 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-24T23:07:27.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:27.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:27.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] boot 2024-04-24T23:07:27.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:07:27.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi150 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi150 to 17088M 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:27.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:27.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:27.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: osd.1 [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] boot 2024-04-24T23:07:27.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: osdmap e13: 2 total, 2 up, 2 in 2024-04-24T23:07:27.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:28.053 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.049+0000 7f0eefea9700 1 -- 172.21.15.187:0/2502589814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8107f80 msgr2=0x7f0ee810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.049+0000 7f0eefea9700 1 --2- 172.21.15.187:0/2502589814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8107f80 0x7f0ee810a3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000b0a0 tx=0x7f0ee000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.049+0000 7f0eefea9700 1 -- 172.21.15.187:0/2502589814 shutdown_connections 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.049+0000 7f0eefea9700 1 --2- 172.21.15.187:0/2502589814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8107f80 0x7f0ee810a3f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.049+0000 7f0eefea9700 1 --2- 172.21.15.187:0/2502589814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee8102d20 0x7f0ee8105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.049+0000 7f0eefea9700 1 --2- 172.21.15.187:0/2502589814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8105650 0x7f0ee8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.049+0000 7f0eefea9700 1 -- 172.21.15.187:0/2502589814 >> 172.21.15.187:0/2502589814 conn(0x7f0ee80fc730 msgr2=0x7f0ee80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 -- 172.21.15.187:0/2502589814 shutdown_connections 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 -- 172.21.15.187:0/2502589814 wait complete. 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 Processor -- start 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 -- start start 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8102d20 0x7f0ee81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:28.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee8105650 0x7f0ee8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8107f80 0x7f0ee819fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee810dfa0 con 0x7f0ee8105650 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee810de20 con 0x7f0ee8107f80 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eefea9700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee810e120 con 0x7f0ee8102d20 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eedc45700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8102d20 0x7f0ee81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eedc45700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8102d20 0x7f0ee81984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:51160/0 (socket says 172.21.15.187:51160) 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.050+0000 7f0eedc45700 1 -- 172.21.15.187:0/1470243972 learned_addr learned my addr 172.21.15.187:0/1470243972 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eed444700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee8105650 0x7f0ee8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eee446700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8107f80 0x7f0ee819fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eedc45700 1 -- 172.21.15.187:0/1470243972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8107f80 msgr2=0x7f0ee819fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eedc45700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8107f80 0x7f0ee819fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eedc45700 1 -- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee8105650 msgr2=0x7f0ee8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:28.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eedc45700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee8105650 0x7f0ee8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:28.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eedc45700 1 -- 172.21.15.187:0/1470243972 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee81a0150 con 0x7f0ee8102d20 2024-04-24T23:07:28.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eedc45700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8102d20 0x7f0ee81984f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400d980 tx=0x7f0ee400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:28.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0edeffd700 1 -- 172.21.15.187:0/1470243972 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4016070 con 0x7f0ee8102d20 2024-04-24T23:07:28.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee81a0440 con 0x7f0ee8102d20 2024-04-24T23:07:28.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0edeffd700 1 -- 172.21.15.187:0/1470243972 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0ee4009dd0 con 0x7f0ee8102d20 2024-04-24T23:07:28.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0edeffd700 1 -- 172.21.15.187:0/1470243972 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400f620 con 0x7f0ee8102d20 2024-04-24T23:07:28.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.051+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee81a09d0 con 0x7f0ee8102d20 2024-04-24T23:07:28.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.052+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee8068f20 con 0x7f0ee8102d20 2024-04-24T23:07:28.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.053+0000 7f0edeffd700 1 -- 172.21.15.187:0/1470243972 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f0ee4019070 con 0x7f0ee8102d20 2024-04-24T23:07:28.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.054+0000 7f0edeffd700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0ed40a2b90 0x7f0ed40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:28.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.054+0000 7f0edeffd700 1 -- 172.21.15.187:0/1470243972 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f0ee40cd250 con 0x7f0ee8102d20 2024-04-24T23:07:28.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.054+0000 7f0eed444700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0ed40a2b90 0x7f0ed40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:28.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.055+0000 7f0eed444700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0ed40a2b90 0x7f0ed40a5050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8004630 tx=0x7f0ed800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:28.060 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.058+0000 7f0edeffd700 1 -- 172.21.15.187:0/1470243972 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0ee409ba00 con 0x7f0ee8102d20 2024-04-24T23:07:28.265 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:28.263+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0ee8062730 con 0x7f0ed40a2b90 2024-04-24T23:07:28.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:28.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:28.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:28.735 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:28.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:28.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:28.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:28.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:28.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:29.485 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:29 smithi187 ceph-mon[33292]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:29.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:29 smithi187 ceph-mon[33292]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:07:29.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:29 smithi187 ceph-mon[33292]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:29.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:29 smithi150 ceph-mon[32247]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:29.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:29 smithi150 ceph-mon[32247]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:07:29.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:29 smithi150 ceph-mon[32247]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:29.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:29 smithi112 ceph-mon[29186]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:29.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:29 smithi112 ceph-mon[29186]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:07:29.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:29 smithi112 ceph-mon[29186]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:31 smithi112 ceph-mon[29186]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:31 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/2917766818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]: dispatch 2024-04-24T23:07:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:31 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]: dispatch 2024-04-24T23:07:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:31 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]': finished 2024-04-24T23:07:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:31 smithi112 ceph-mon[29186]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:07:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:32.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:31 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/3343373113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:31 smithi150 ceph-mon[32247]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:31 smithi150 ceph-mon[32247]: from='client.? 172.21.15.187:0/2917766818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]: dispatch 2024-04-24T23:07:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:31 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]: dispatch 2024-04-24T23:07:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:31 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]': finished 2024-04-24T23:07:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:31 smithi150 ceph-mon[32247]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:07:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:32.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:31 smithi150 ceph-mon[32247]: from='client.? 172.21.15.187:0/3343373113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:31 smithi187 ceph-mon[33292]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:31 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/2917766818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]: dispatch 2024-04-24T23:07:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:31 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]: dispatch 2024-04-24T23:07:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:31 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4899db4a-7c50-4d57-bce9-50d0e840ca6f"}]': finished 2024-04-24T23:07:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:31 smithi187 ceph-mon[33292]: osdmap e15: 3 total, 2 up, 3 in 2024-04-24T23:07:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:31 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/3343373113' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:34.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:33 smithi150 ceph-mon[32247]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:33 smithi187 ceph-mon[33292]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:34.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:33 smithi112 ceph-mon[29186]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:35.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:35 smithi187 ceph-mon[33292]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:35 smithi150 ceph-mon[32247]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:36.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:35 smithi112 ceph-mon[29186]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:37.815 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:37 smithi187 ceph-mon[33292]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:38.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:37 smithi150 ceph-mon[32247]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:38.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:37 smithi112 ceph-mon[29186]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:38.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:38 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:07:38.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:38 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:38.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:38 smithi187 ceph-mon[33292]: Deploying daemon osd.2 on smithi187 2024-04-24T23:07:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:38 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:07:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:38 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:38 smithi150 ceph-mon[32247]: Deploying daemon osd.2 on smithi187 2024-04-24T23:07:39.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:38 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:07:39.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:38 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:39.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:38 smithi112 ceph-mon[29186]: Deploying daemon osd.2 on smithi187 2024-04-24T23:07:39.840 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:39 smithi187 ceph-mon[33292]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:39 smithi150 ceph-mon[32247]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:40.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:39 smithi112 ceph-mon[29186]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:42.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:41 smithi150 ceph-mon[32247]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:42.102 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:41 smithi187 ceph-mon[33292]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:42.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:41 smithi112 ceph-mon[29186]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:44.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:43 smithi187 ceph-mon[33292]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:44.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:43 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:44.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:43 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:44.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:43 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:43 smithi112 ceph-mon[29186]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:43 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:43 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:43 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:43 smithi150 ceph-mon[32247]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:43 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:43 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:43 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:46.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:45 smithi187 ceph-mon[33292]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:46.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:45 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:45 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:45 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:46.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:45 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:46.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:45 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:45 smithi112 ceph-mon[29186]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:45 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:45 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:45 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:45 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:45 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.280 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.277+0000 7f0edeffd700 1 -- 172.21.15.187:0/1470243972 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0ee8062730 con 0x7f0ed40a2b90 2024-04-24T23:07:46.280 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 2 on host 'smithi187' 2024-04-24T23:07:46.281 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0ed40a2b90 msgr2=0x7f0ed40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0ed40a2b90 0x7f0ed40a5050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8004630 tx=0x7f0ed800b3c0 comp rx=0 tx=0).stop 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8102d20 msgr2=0x7f0ee81984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8102d20 0x7f0ee81984f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400d980 tx=0x7f0ee400dc90 comp rx=0 tx=0).stop 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 shutdown_connections 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0ed40a2b90 0x7f0ed40a5050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ee8107f80 0x7f0ee819fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ee8102d20 0x7f0ee81984f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.279+0000 7f0eefea9700 1 --2- 172.21.15.187:0/1470243972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ee8105650 0x7f0ee8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.280+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 >> 172.21.15.187:0/1470243972 conn(0x7f0ee80fc730 msgr2=0x7f0ee8103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.280+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 shutdown_connections 2024-04-24T23:07:46.282 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:07:46.280+0000 7f0eefea9700 1 -- 172.21.15.187:0/1470243972 wait complete. 2024-04-24T23:07:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:45 smithi150 ceph-mon[32247]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:45 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:45 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:45 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:45 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:45 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:46.877 DEBUG:teuthology.orchestra.run.smithi187:osd.2> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T23:07:46.920 INFO:tasks.cephadm:Deploying osd.3 on smithi112 with /dev/vg_nvme/lv_3... 2024-04-24T23:07:46.920 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:07:47.165 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:07:47.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:47.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:47.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:47.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:47 smithi187 ceph-mon[33292]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:47.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:47 smithi187 ceph-mon[33292]: from='osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:07:47.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:47 smithi187 ceph-mon[33292]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:07:47.511 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:47.511 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:47.511 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:47.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:47 smithi112 ceph-mon[29186]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:47.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:47 smithi112 ceph-mon[29186]: from='osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:07:47.512 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:47 smithi112 ceph-mon[29186]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:07:47.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:47.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:47.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:47.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:47 smithi150 ceph-mon[32247]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:47.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:47 smithi150 ceph-mon[32247]: from='osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:07:47.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:47 smithi150 ceph-mon[32247]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:07:48.388 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:48 smithi187 ceph-mon[33292]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:07:48.389 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:48 smithi187 ceph-mon[33292]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:07:48.389 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:48 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:48.389 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:48 smithi187 ceph-mon[33292]: from='osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:07:48.389 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:48 smithi187 ceph-mon[33292]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:07:48.526 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:48 smithi112 ceph-mon[29186]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:07:48.526 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:48 smithi112 ceph-mon[29186]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:07:48.526 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:48 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:48.526 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:48 smithi112 ceph-mon[29186]: from='osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:07:48.526 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:48 smithi112 ceph-mon[29186]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:07:48.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:48 smithi150 ceph-mon[32247]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:07:48.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:48 smithi150 ceph-mon[32247]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:07:48.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:48 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:48.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:48 smithi150 ceph-mon[32247]: from='osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:07:48.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:48 smithi150 ceph-mon[32247]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:07:49.061 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:07:49.062 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:07:49.062 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-24T23:07:49.062 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-24T23:07:49.062 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118234 s, 887 MB/s 2024-04-24T23:07:49.062 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-24T23:07:49.209 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:48 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2[38563]: 2024-04-24T23:07:48.956+0000 7f46cdf9f700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:07:49.209 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:48 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2[38563]: 2024-04-24T23:07:48.962+0000 7f46c5d8a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:49.364 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:49 smithi112 ceph-mon[29186]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:07:49.364 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:49 smithi112 ceph-mon[29186]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:07:49.364 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:49.365 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:49.365 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:49 smithi112 ceph-mon[29186]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:49.365 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:49 smithi112 ceph-mon[29186]: from='osd.2 ' entity='osd.2' 2024-04-24T23:07:49.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:49 smithi187 ceph-mon[33292]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:07:49.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:49 smithi187 ceph-mon[33292]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:07:49.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:49.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:49.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:49 smithi187 ceph-mon[33292]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:49.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:49 smithi187 ceph-mon[33292]: from='osd.2 ' entity='osd.2' 2024-04-24T23:07:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:49 smithi150 ceph-mon[32247]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:07:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:49 smithi150 ceph-mon[32247]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:07:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:49 smithi150 ceph-mon[32247]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-24T23:07:49.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:49 smithi150 ceph-mon[32247]: from='osd.2 ' entity='osd.2' 2024-04-24T23:07:49.658 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-04-24T23:07:49.899 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:07:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:07:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:07:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi187 2024-04-24T23:07:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi187 to 17088M 2024-04-24T23:07:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] boot 2024-04-24T23:07:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:07:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:50.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:07:50.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:07:50.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:50.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi187 2024-04-24T23:07:50.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi187 to 17088M 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] boot 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:07:50.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:07:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:07:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi187 2024-04-24T23:07:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:50.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi187 to 17088M 2024-04-24T23:07:50.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:50.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:50.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:50.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: osd.2 [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] boot 2024-04-24T23:07:50.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: osdmap e18: 3 total, 3 up, 3 in 2024-04-24T23:07:50.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:51.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:51 smithi112 ceph-mon[29186]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:51.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:51 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:07:51.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:51 smithi150 ceph-mon[32247]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:51.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:51 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:07:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 -- 172.21.15.112:0/2728542479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520101190 msgr2=0x7f95201015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:51.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 --2- 172.21.15.112:0/2728542479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520101190 0x7f95201015b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9510009920 tx=0x7f9510009c30 comp rx=0 tx=0).stop 2024-04-24T23:07:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 -- 172.21.15.112:0/2728542479 shutdown_connections 2024-04-24T23:07:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 --2- 172.21.15.112:0/2728542479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9520102d30 0x7f95201095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 --2- 172.21.15.112:0/2728542479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520101190 0x7f95201015b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 --2- 172.21.15.112:0/2728542479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9520102390 0x7f95201027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 -- 172.21.15.112:0/2728542479 >> 172.21.15.112:0/2728542479 conn(0x7f95200fc730 msgr2=0x7f95200feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 -- 172.21.15.112:0/2728542479 shutdown_connections 2024-04-24T23:07:51.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.568+0000 7f9524da7700 1 -- 172.21.15.112:0/2728542479 wait complete. 2024-04-24T23:07:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f9524da7700 1 Processor -- start 2024-04-24T23:07:51.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f9524da7700 1 -- start start 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f9524da7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9520101190 0x7f9520198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f9524da7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520102390 0x7f9520198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f9524da7700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9520102d30 0x7f952019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f9524da7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f952010ced0 con 0x7f9520102390 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f9524da7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f952010cd50 con 0x7f9520101190 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f9524da7700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f952010d050 con 0x7f9520102d30 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f951dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520102390 0x7f9520198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f951dd9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520102390 0x7f9520198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51478/0 (socket says 172.21.15.112:51478) 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.569+0000 7f951dd9b700 1 -- 172.21.15.112:0/1360537948 learned_addr learned my addr 172.21.15.112:0/1360537948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f951e59c700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9520101190 0x7f9520198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f951ed9d700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9520102d30 0x7f952019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:51.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f951dd9b700 1 -- 172.21.15.112:0/1360537948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9520102d30 msgr2=0x7f952019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f951dd9b700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9520102d30 0x7f952019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f951dd9b700 1 -- 172.21.15.112:0/1360537948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9520101190 msgr2=0x7f9520198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f951dd9b700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9520101190 0x7f9520198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f951dd9b700 1 -- 172.21.15.112:0/1360537948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95201a0280 con 0x7f9520102390 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f951dd9b700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520102390 0x7f9520198b60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f950800e9f0 tx=0x7f950800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f95177fe700 1 -- 172.21.15.112:0/1360537948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950800cbd0 con 0x7f9520102390 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95201a0570 con 0x7f9520102390 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f95177fe700 1 -- 172.21.15.112:0/1360537948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f950800cd70 con 0x7f9520102390 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f95177fe700 1 -- 172.21.15.112:0/1360537948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508010430 con 0x7f9520102390 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.570+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95201a0b00 con 0x7f9520102390 2024-04-24T23:07:51.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.571+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9520068f20 con 0x7f9520102390 2024-04-24T23:07:51.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.572+0000 7f95177fe700 1 -- 172.21.15.112:0/1360537948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f9508004e50 con 0x7f9520102390 2024-04-24T23:07:51.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.573+0000 7f95177fe700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f950c0a2b90 0x7f950c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.573+0000 7f951e59c700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f950c0a2b90 0x7f950c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.573+0000 7f95177fe700 1 -- 172.21.15.112:0/1360537948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f95080cdcf0 con 0x7f9520102390 2024-04-24T23:07:51.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.573+0000 7f951e59c700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f950c0a2b90 0x7f950c0a5050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f951000a010 tx=0x7f9510019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:51.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.578+0000 7f95177fe700 1 -- 172.21.15.112:0/1360537948 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f950809c110 con 0x7f9520102390 2024-04-24T23:07:51.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:51 smithi187 ceph-mon[33292]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:51.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:51 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:07:51.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:51.782+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f952006b070 con 0x7f950c0a2b90 2024-04-24T23:07:52.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:52 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:07:52.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:52 smithi112 ceph-mon[29186]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:07:52.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:52 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:07:52.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:52 smithi112 ceph-mon[29186]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:52.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:52 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:52.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:52 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:52.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:52 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:52.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:52 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:07:52.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:52 smithi150 ceph-mon[32247]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:07:52.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:52 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:07:52.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:52 smithi150 ceph-mon[32247]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:52.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:52 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:52.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:52 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:52.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:52 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:52.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:52 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-24T23:07:52.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:52 smithi187 ceph-mon[33292]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:07:52.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:52 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:07:52.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:52 smithi187 ceph-mon[33292]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:52.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:52 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:52.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:52 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:52.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:52 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:52.867 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:52 smithi112 sudo[48745]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:07:52.867 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:52 smithi112 sudo[48745]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:52.867 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:52 smithi112 sudo[48745]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:52.867 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:52 smithi112 sudo[48745]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:53.278 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:52 smithi112 sudo[48761]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:07:53.279 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:52 smithi112 sudo[48761]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:53.279 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:52 smithi112 sudo[48761]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:53.279 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:07:52 smithi112 sudo[48761]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:53.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:07:53.285+0000 7f95177fe700 1 -- 172.21.15.112:0/1360537948 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9508098080 con 0x7f9520102390 2024-04-24T23:07:53.391 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:53 smithi150 sudo[40815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:07:53.392 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:53 smithi150 sudo[40815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:53.392 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:53 smithi150 sudo[40815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:53.392 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:53 smithi150 sudo[40815]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:53.392 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:07:53.392 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:53 smithi150 ceph-mon[32247]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:07:53.392 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:53 smithi150 ceph-mon[32247]: pgmap v83: 1 pgs: 1 active+clean; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:53.392 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:53.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:07:53.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:53 smithi112 ceph-mon[29186]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:07:53.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:53 smithi112 ceph-mon[29186]: pgmap v83: 1 pgs: 1 active+clean; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:53.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:53.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:07:53.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:53 smithi187 ceph-mon[33292]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:07:53.659 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:53 smithi187 ceph-mon[33292]: pgmap v83: 1 pgs: 1 active+clean; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:53.660 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:53.818 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:53 smithi150 sudo[40818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:07:53.818 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:53 smithi150 sudo[40818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:53.818 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:53 smithi150 sudo[40818]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:53.818 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:07:53 smithi150 sudo[40818]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:53.912 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:53 smithi187 sudo[42206]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:07:53.912 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:53 smithi187 sudo[42206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:53.912 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:53 smithi187 sudo[42206]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:53.913 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:53 smithi187 sudo[42206]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:54.235 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:53 smithi187 sudo[42209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:07:54.236 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:53 smithi187 sudo[42209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:54.236 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:53 smithi187 sudo[42209]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:54.236 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:07:53 smithi187 sudo[42209]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:54.446 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 sudo[48817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:07:54.446 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 sudo[48817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:54.446 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 sudo[48817]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:54.446 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 sudo[48817]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 ceph-mon[33292]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 ceph-mon[33292]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 ceph-mon[33292]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 sudo[42212]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 sudo[42212]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 sudo[42212]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:54.706 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 sudo[42212]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 sudo[48838]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 sudo[48838]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 sudo[48838]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 sudo[48838]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 ceph-mon[29186]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 ceph-mon[29186]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:54.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:54 smithi112 ceph-mon[29186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:54.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:54 smithi150 ceph-mon[32247]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-24T23:07:54.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:54 smithi150 ceph-mon[32247]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:07:54.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:54 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:54.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:54 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:07:54.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:54 smithi150 ceph-mon[32247]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:54.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 sudo[42215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:07:54.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 sudo[42215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:54.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 sudo[42215]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:54.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:54 smithi187 sudo[42215]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:55.439 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 sudo[40821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:07:55.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 sudo[40821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:55.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 sudo[40821]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:55.440 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 sudo[40821]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1086874197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c500d4f-a92d-489a-98c8-758763754114"}]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: pgmap v85: 1 pgs: 1 active+clean; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1086874197' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c500d4f-a92d-489a-98c8-758763754114"}]': finished 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:55 smithi187 ceph-mon[33292]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:55.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:55.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:55.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1086874197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c500d4f-a92d-489a-98c8-758763754114"}]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: pgmap v85: 1 pgs: 1 active+clean; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1086874197' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c500d4f-a92d-489a-98c8-758763754114"}]': finished 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.781 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:55 smithi112 ceph-mon[29186]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:55.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 sudo[40824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:07:55.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 sudo[40824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:07:55.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 sudo[40824]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:07:55.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 sudo[40824]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:07:55.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1086874197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c500d4f-a92d-489a-98c8-758763754114"}]: dispatch 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: pgmap v85: 1 pgs: 1 active+clean; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1086874197' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c500d4f-a92d-489a-98c8-758763754114"}]': finished 2024-04-24T23:07:55.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: osdmap e22: 4 total, 3 up, 4 in 2024-04-24T23:07:55.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:07:55.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:07:55.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-24T23:07:55.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-24T23:07:55.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-24T23:07:55.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:55 smithi150 ceph-mon[32247]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:07:56.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:56 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/287016726' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:56.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:56 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/287016726' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:56.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:56 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/287016726' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:57.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:57 smithi187 ceph-mon[33292]: pgmap v87: 1 pgs: 1 active+clean; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:57.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:57 smithi112 ceph-mon[29186]: pgmap v87: 1 pgs: 1 active+clean; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:07:57.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:57 smithi150 ceph-mon[32247]: pgmap v87: 1 pgs: 1 active+clean; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:08:00.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:07:59 smithi150 ceph-mon[32247]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 16 KiB/s rd, 490 KiB/s wr, 44 op/s 2024-04-24T23:08:00.075 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:07:59 smithi112 ceph-mon[29186]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 16 KiB/s rd, 490 KiB/s wr, 44 op/s 2024-04-24T23:08:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:07:59 smithi187 ceph-mon[33292]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 16 KiB/s rd, 490 KiB/s wr, 44 op/s 2024-04-24T23:08:02.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:01 smithi112 ceph-mon[29186]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 367 KiB/s wr, 33 op/s 2024-04-24T23:08:02.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:01 smithi150 ceph-mon[32247]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 367 KiB/s wr, 33 op/s 2024-04-24T23:08:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:01 smithi187 ceph-mon[33292]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 12 KiB/s rd, 367 KiB/s wr, 33 op/s 2024-04-24T23:08:03.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:02 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:08:03.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:02 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:03.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:02 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:08:03.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:02 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:03.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:02 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:08:03.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:02 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:04.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:03 smithi112 ceph-mon[29186]: Deploying daemon osd.3 on smithi112 2024-04-24T23:08:04.001 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:03 smithi112 ceph-mon[29186]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 311 KiB/s wr, 28 op/s 2024-04-24T23:08:04.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:03 smithi150 ceph-mon[32247]: Deploying daemon osd.3 on smithi112 2024-04-24T23:08:04.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:03 smithi150 ceph-mon[32247]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 311 KiB/s wr, 28 op/s 2024-04-24T23:08:04.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:03 smithi187 ceph-mon[33292]: Deploying daemon osd.3 on smithi112 2024-04-24T23:08:04.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:03 smithi187 ceph-mon[33292]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 311 KiB/s wr, 28 op/s 2024-04-24T23:08:06.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:05 smithi112 ceph-mon[29186]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 9.5 KiB/s rd, 294 KiB/s wr, 26 op/s 2024-04-24T23:08:06.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:05 smithi150 ceph-mon[32247]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 9.5 KiB/s rd, 294 KiB/s wr, 26 op/s 2024-04-24T23:08:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:05 smithi187 ceph-mon[33292]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 9.5 KiB/s rd, 294 KiB/s wr, 26 op/s 2024-04-24T23:08:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:07 smithi112 ceph-mon[29186]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 8.1 KiB/s rd, 249 KiB/s wr, 22 op/s 2024-04-24T23:08:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:08.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:07 smithi187 ceph-mon[33292]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 8.1 KiB/s rd, 249 KiB/s wr, 22 op/s 2024-04-24T23:08:08.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:08.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:08.061 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:08.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:07 smithi150 ceph-mon[32247]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 8.1 KiB/s rd, 249 KiB/s wr, 22 op/s 2024-04-24T23:08:08.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:08.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:08.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:10.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:09 smithi112 ceph-mon[29186]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 7.9 KiB/s rd, 245 KiB/s wr, 22 op/s 2024-04-24T23:08:10.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:09 smithi150 ceph-mon[32247]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 7.9 KiB/s rd, 245 KiB/s wr, 22 op/s 2024-04-24T23:08:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:09 smithi187 ceph-mon[33292]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 7.9 KiB/s rd, 245 KiB/s wr, 22 op/s 2024-04-24T23:08:11.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:11.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:11.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.461 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:11 smithi112 ceph-mon[29186]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:08:11.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.537+0000 7f95177fe700 1 -- 172.21.15.112:0/1360537948 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f952006b070 con 0x7f950c0a2b90 2024-04-24T23:08:11.540 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 3 on host 'smithi112' 2024-04-24T23:08:11.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f950c0a2b90 msgr2=0x7f950c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:11.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f950c0a2b90 0x7f950c0a5050 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f951000a010 tx=0x7f9510019040 comp rx=0 tx=0).stop 2024-04-24T23:08:11.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520102390 msgr2=0x7f9520198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:11.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520102390 0x7f9520198b60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f950800e9f0 tx=0x7f950800ed00 comp rx=0 tx=0).stop 2024-04-24T23:08:11.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 shutdown_connections 2024-04-24T23:08:11.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f950c0a2b90 0x7f950c0a5050 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:11.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9520101190 0x7f9520198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:11.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9520102d30 0x7f952019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:11.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 --2- 172.21.15.112:0/1360537948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9520102390 0x7f9520198b60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:11.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 >> 172.21.15.112:0/1360537948 conn(0x7f95200fc730 msgr2=0x7f952006a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:11.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 shutdown_connections 2024-04-24T23:08:11.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:08:11.540+0000 7f9524da7700 1 -- 172.21.15.112:0/1360537948 wait complete. 2024-04-24T23:08:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:11 smithi187 ceph-mon[33292]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:08:11.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:11.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:11.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:11.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:11 smithi150 ceph-mon[32247]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:08:12.148 DEBUG:teuthology.orchestra.run.smithi112:osd.3> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T23:08:12.150 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_3... 2024-04-24T23:08:12.151 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:08:12.397 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:08:12.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:12 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:12.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:12 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:12.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:12 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:12.763 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:12 smithi150 ceph-mon[32247]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:08:12.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:12 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:12.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:12 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:12.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:12 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:12.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:12 smithi112 ceph-mon[29186]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:08:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:12 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:12 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:12 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:12 smithi187 ceph-mon[33292]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:08:13.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:13 smithi112 ceph-mon[29186]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:08:13.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:13 smithi112 ceph-mon[29186]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:08:13.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:13 smithi112 ceph-mon[29186]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:08:13.606 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:13.607 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:13 smithi112 ceph-mon[29186]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:08:13.607 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:08:13 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3[50879]: 2024-04-24T23:08:13.503+0000 7f91ca485700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:08:13.607 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:08:13 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3[50879]: 2024-04-24T23:08:13.510+0000 7f91c6278700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:13.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:13 smithi150 ceph-mon[32247]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:08:13.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:13 smithi150 ceph-mon[32247]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:08:13.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:13 smithi150 ceph-mon[32247]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:08:13.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:13.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:13 smithi150 ceph-mon[32247]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:08:13.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:13 smithi187 ceph-mon[33292]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:08:13.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:13 smithi187 ceph-mon[33292]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:08:13.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:13 smithi187 ceph-mon[33292]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:08:13.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:13.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:13 smithi187 ceph-mon[33292]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-24T23:08:14.364 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:08:14.364 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:08:14.364 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-24T23:08:14.364 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-24T23:08:14.364 INFO:teuthology.orchestra.run.smithi150.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112805 s, 930 MB/s 2024-04-24T23:08:14.364 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-24T23:08:14.707 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:14 smithi150 ceph-mon[32247]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:08:14.707 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:14 smithi150 ceph-mon[32247]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:08:14.707 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:14.707 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:14.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:14 smithi150 ceph-mon[32247]: osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] boot 2024-04-24T23:08:14.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:14 smithi150 ceph-mon[32247]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:08:14.708 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:14.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:14 smithi112 ceph-mon[29186]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:08:14.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:14 smithi112 ceph-mon[29186]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:08:14.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:14.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:14.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:14 smithi112 ceph-mon[29186]: osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] boot 2024-04-24T23:08:14.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:14 smithi112 ceph-mon[29186]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:08:14.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:14 smithi187 ceph-mon[33292]: from='osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:08:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:14 smithi187 ceph-mon[33292]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:08:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:14 smithi187 ceph-mon[33292]: osd.3 [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] boot 2024-04-24T23:08:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:14 smithi187 ceph-mon[33292]: osdmap e25: 4 total, 4 up, 4 in 2024-04-24T23:08:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:15.034 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-04-24T23:08:15.275 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:08:15.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:08:15.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:08:15.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:15.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:08:15.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:15.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:15.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:15.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:15.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:15.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:15.780 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:08:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:08:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:08:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:15.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:15.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:15.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:15.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:16.868 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:16 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi112 2024-04-24T23:08:16.868 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:16 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-24T23:08:16.868 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:16 smithi150 ceph-mon[32247]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:08:16.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 -- 172.21.15.150:0/2396273435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0102290 msgr2=0x7fdcd01026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:16.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 --2- 172.21.15.150:0/2396273435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0102290 0x7fdcd01026b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00099c0 tx=0x7fdcc0009cd0 comp rx=0 tx=0).stop 2024-04-24T23:08:16.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 -- 172.21.15.150:0/2396273435 shutdown_connections 2024-04-24T23:08:16.976 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 --2- 172.21.15.150:0/2396273435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0102290 0x7fdcd01026b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 --2- 172.21.15.150:0/2396273435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd0103e30 0x7fdcd010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 --2- 172.21.15.150:0/2396273435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0103490 0x7fdcd01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 -- 172.21.15.150:0/2396273435 >> 172.21.15.150:0/2396273435 conn(0x7fdcd00fd830 msgr2=0x7fdcd00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 -- 172.21.15.150:0/2396273435 shutdown_connections 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.971+0000 7fdcd5418700 1 -- 172.21.15.150:0/2396273435 wait complete. 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.972+0000 7fdcd5418700 1 Processor -- start 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.972+0000 7fdcd5418700 1 -- start start 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.972+0000 7fdcd5418700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd0102290 0x7fdcd01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.972+0000 7fdcd5418700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0103490 0x7fdcd0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.972+0000 7fdcd5418700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0103e30 0x7fdcd019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:16.977 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.972+0000 7fdcd5418700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010e010 con 0x7fdcd0103490 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.972+0000 7fdcd5418700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010de90 con 0x7fdcd0103e30 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.972+0000 7fdcd5418700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd010e190 con 0x7fdcd0102290 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0103e30 0x7fdcd019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0103e30 0x7fdcd019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:39616/0 (socket says 172.21.15.150:39616) 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 -- 172.21.15.150:0/3406970307 learned_addr learned my addr 172.21.15.150:0/3406970307 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdcceffd700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd0102290 0x7fdcd01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdcce7fc700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0103490 0x7fdcd0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 -- 172.21.15.150:0/3406970307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd0102290 msgr2=0x7fdcd01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd0102290 0x7fdcd01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:16.978 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 -- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0103490 msgr2=0x7fdcd0198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0103490 0x7fdcd0198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 -- 172.21.15.150:0/3406970307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd00744e0 con 0x7fdcd0103e30 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdcce7fc700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0103490 0x7fdcd0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdccf7fe700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0103e30 0x7fdcd019fbb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800ada0 tx=0x7fdcc800c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdcb7fff700 1 -- 172.21.15.150:0/3406970307 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8017020 con 0x7fdcd0103e30 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd00747d0 con 0x7fdcd0103e30 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdcb7fff700 1 -- 172.21.15.150:0/3406970307 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdcc800a0d0 con 0x7fdcd0103e30 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdcb7fff700 1 -- 172.21.15.150:0/3406970307 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80054e0 con 0x7fdcd0103e30 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.973+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd0074cc0 con 0x7fdcd0103e30 2024-04-24T23:08:16.979 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.974+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd0068f20 con 0x7fdcd0103e30 2024-04-24T23:08:16.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.976+0000 7fdcb7fff700 1 -- 172.21.15.150:0/3406970307 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdcc8004ad0 con 0x7fdcd0103e30 2024-04-24T23:08:16.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.976+0000 7fdcb7fff700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdcb80a2c20 0x7fdcb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:16.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.976+0000 7fdcb7fff700 1 -- 172.21.15.150:0/3406970307 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fdcc80cdbf0 con 0x7fdcd0103e30 2024-04-24T23:08:16.980 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.976+0000 7fdcceffd700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdcb80a2c20 0x7fdcb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:16.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.977+0000 7fdcceffd700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdcb80a2c20 0x7fdcb80a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0003990 tx=0x7fdcc0005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:16.981 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:16.980+0000 7fdcb7fff700 1 -- 172.21.15.150:0/3406970307 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdcc8021020 con 0x7fdcd0103e30 2024-04-24T23:08:17.185 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:17.183+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdcd01091e0 con 0x7fdcb80a2c20 2024-04-24T23:08:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:16 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi112 2024-04-24T23:08:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:16 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-24T23:08:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:16 smithi187 ceph-mon[33292]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:08:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:16 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi112 2024-04-24T23:08:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:16 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-24T23:08:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:16 smithi112 ceph-mon[29186]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:08:18.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:17 smithi112 ceph-mon[29186]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:18.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:17 smithi112 ceph-mon[29186]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:08:18.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:17 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:18.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:17 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:18.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:17 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:18.053 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:17 smithi150 ceph-mon[32247]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:18.054 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:17 smithi150 ceph-mon[32247]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:08:18.054 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:17 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:18.054 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:17 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:18.054 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:17 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:17 smithi187 ceph-mon[33292]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:17 smithi187 ceph-mon[33292]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:08:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:17 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:17 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:17 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:19.104 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:18 smithi150 ceph-mon[32247]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:19.104 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:18 smithi150 ceph-mon[32247]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:08:19.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:18 smithi187 ceph-mon[33292]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:19.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:18 smithi187 ceph-mon[33292]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:08:19.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:18 smithi112 ceph-mon[29186]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:19.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:18 smithi112 ceph-mon[29186]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:08:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:19 smithi187 ceph-mon[33292]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:08:20.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:19 smithi112 ceph-mon[29186]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:08:20.296 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:19 smithi150 ceph-mon[32247]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:08:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:20 smithi187 ceph-mon[33292]: from='client.? 172.21.15.150:0/2511327102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]: dispatch 2024-04-24T23:08:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:20 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]: dispatch 2024-04-24T23:08:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:20 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]': finished 2024-04-24T23:08:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:20 smithi187 ceph-mon[33292]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:08:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:20 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:20 smithi187 ceph-mon[33292]: from='client.? 172.21.15.150:0/2918244220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:20 smithi112 ceph-mon[29186]: from='client.? 172.21.15.150:0/2511327102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]: dispatch 2024-04-24T23:08:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:20 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]: dispatch 2024-04-24T23:08:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:20 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]': finished 2024-04-24T23:08:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:20 smithi112 ceph-mon[29186]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:08:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:20 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:20 smithi112 ceph-mon[29186]: from='client.? 172.21.15.150:0/2918244220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:21.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:20 smithi150 ceph-mon[32247]: from='client.? 172.21.15.150:0/2511327102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]: dispatch 2024-04-24T23:08:21.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:20 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]: dispatch 2024-04-24T23:08:21.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:20 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57a393a7-46ea-440d-b990-315d5806e675"}]': finished 2024-04-24T23:08:21.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:20 smithi150 ceph-mon[32247]: osdmap e30: 5 total, 4 up, 5 in 2024-04-24T23:08:21.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:20 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:21.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:20 smithi150 ceph-mon[32247]: from='client.? 172.21.15.150:0/2918244220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:21 smithi187 ceph-mon[33292]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-24T23:08:22.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:21 smithi112 ceph-mon[29186]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-24T23:08:22.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:21 smithi150 ceph-mon[32247]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-24T23:08:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:23 smithi187 ceph-mon[33292]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:08:24.249 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:23 smithi150 ceph-mon[32247]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:08:24.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:23 smithi112 ceph-mon[29186]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:08:26.121 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:25 smithi150 ceph-mon[32247]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-24T23:08:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:25 smithi187 ceph-mon[33292]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-24T23:08:26.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:25 smithi112 ceph-mon[29186]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-24T23:08:28.191 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:27 smithi150 ceph-mon[32247]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:28.191 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:08:28.191 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:27 smithi187 ceph-mon[33292]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:08:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:28.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:27 smithi112 ceph-mon[29186]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:08:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:29.216 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:28 smithi150 ceph-mon[32247]: Deploying daemon osd.4 on smithi150 2024-04-24T23:08:29.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:28 smithi187 ceph-mon[33292]: Deploying daemon osd.4 on smithi150 2024-04-24T23:08:29.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:28 smithi112 ceph-mon[29186]: Deploying daemon osd.4 on smithi150 2024-04-24T23:08:29.968 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:29 smithi150 ceph-mon[32247]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:29 smithi187 ceph-mon[33292]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:30.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:29 smithi112 ceph-mon[29186]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:31 smithi187 ceph-mon[33292]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:32.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:31 smithi150 ceph-mon[32247]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:31 smithi112 ceph-mon[29186]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:34.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:33 smithi150 ceph-mon[32247]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:34.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:33 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:34.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:33 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:34.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:33 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:33 smithi187 ceph-mon[33292]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:33 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:33 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:33 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:34.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:33 smithi112 ceph-mon[29186]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:34.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:33 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:34.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:33 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:34.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:33 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:36.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:35 smithi112 ceph-mon[29186]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:36.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:35 smithi150 ceph-mon[32247]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:35 smithi187 ceph-mon[33292]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:36.251 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.249+0000 7fdcb7fff700 1 -- 172.21.15.150:0/3406970307 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdcd01091e0 con 0x7fdcb80a2c20 2024-04-24T23:08:36.251 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-04-24T23:08:36.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.251+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdcb80a2c20 msgr2=0x7fdcb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:36.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.251+0000 7fdcd5418700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdcb80a2c20 0x7fdcb80a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0003990 tx=0x7fdcc0005ec0 comp rx=0 tx=0).stop 2024-04-24T23:08:36.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.251+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0103e30 msgr2=0x7fdcd019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:36.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.251+0000 7fdcd5418700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0103e30 0x7fdcd019fbb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800ada0 tx=0x7fdcc800c330 comp rx=0 tx=0).stop 2024-04-24T23:08:36.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.252+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 shutdown_connections 2024-04-24T23:08:36.253 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.252+0000 7fdcd5418700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdcb80a2c20 0x7fdcb80a50e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:36.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.252+0000 7fdcd5418700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdcd0103e30 0x7fdcd019fbb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:36.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.252+0000 7fdcd5418700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcd0102290 0x7fdcd01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:36.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.252+0000 7fdcd5418700 1 --2- 172.21.15.150:0/3406970307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0103490 0x7fdcd0198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:36.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.252+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 >> 172.21.15.150:0/3406970307 conn(0x7fdcd00fd830 msgr2=0x7fdcd0106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:36.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.252+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 shutdown_connections 2024-04-24T23:08:36.254 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:08:36.252+0000 7fdcd5418700 1 -- 172.21.15.150:0/3406970307 wait complete. 2024-04-24T23:08:37.145 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T23:08:37.147 INFO:tasks.cephadm:Deploying osd.5 on smithi187 with /dev/vg_nvme/lv_3... 2024-04-24T23:08:37.148 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:08:37.393 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:08:37.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:37.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:37.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:37 smithi187 ceph-mon[33292]: from='osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:37.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:37 smithi187 ceph-mon[33292]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:37.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:37 smithi187 ceph-mon[33292]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:37.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:37.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:37.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:37 smithi112 ceph-mon[29186]: from='osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:37.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:37 smithi112 ceph-mon[29186]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:37.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:37 smithi112 ceph-mon[29186]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:37.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:37.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:37.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:37 smithi150 ceph-mon[32247]: from='osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:37.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:37 smithi150 ceph-mon[32247]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:37.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:37 smithi150 ceph-mon[32247]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:38 smithi112 ceph-mon[29186]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:08:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:38 smithi112 ceph-mon[29186]: from='osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:08:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:38 smithi112 ceph-mon[29186]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:08:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:38 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:38 smithi112 ceph-mon[29186]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:08:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:38 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:38 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:38.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-mon[32247]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:08:38.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-mon[32247]: from='osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:08:38.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-mon[32247]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:08:38.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:38.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-mon[32247]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:08:38.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:38.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:38.569 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:08:38.237+0000 7ffb92d61700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:08:38.569 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:08:38 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:08:38.244+0000 7ffb8eb54700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:38.586 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:38 smithi187 ceph-mon[33292]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:08:38.586 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:38 smithi187 ceph-mon[33292]: from='osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:08:38.586 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:38 smithi187 ceph-mon[33292]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:08:38.586 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:38 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:38.587 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:38 smithi187 ceph-mon[33292]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:08:38.587 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:38 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:38.587 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:38 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:39.293 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:08:39.293 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:08:39.293 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2024-04-24T23:08:39.293 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2024-04-24T23:08:39.293 INFO:teuthology.orchestra.run.smithi187.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011167 s, 939 MB/s 2024-04-24T23:08:39.293 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2024-04-24T23:08:39.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:39 smithi112 ceph-mon[29186]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:08:39.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:39 smithi112 ceph-mon[29186]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:08:39.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:39.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:39 smithi112 ceph-mon[29186]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:39.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:39.545 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:39 smithi150 ceph-mon[32247]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:08:39.545 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:39 smithi150 ceph-mon[32247]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:08:39.545 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:39.545 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:39 smithi150 ceph-mon[32247]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:39.546 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:39.583 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:39 smithi187 ceph-mon[33292]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:08:39.583 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:39 smithi187 ceph-mon[33292]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:08:39.584 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:39.584 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:39 smithi187 ceph-mon[33292]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-04-24T23:08:39.584 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:40.042 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi187:vg_nvme/lv_3 2024-04-24T23:08:40.284 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:08:40.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:40 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:08:40.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:40 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:08:40.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:40 smithi150 ceph-mon[32247]: osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] boot 2024-04-24T23:08:40.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:40 smithi150 ceph-mon[32247]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:08:40.369 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:40 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:40.406 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:40 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:08:40.406 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:40 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:08:40.407 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:40 smithi187 ceph-mon[33292]: osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] boot 2024-04-24T23:08:40.407 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:40 smithi187 ceph-mon[33292]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:08:40.407 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:40 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:40.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:40 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:08:40.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:40 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:08:40.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:40 smithi112 ceph-mon[29186]: osd.4 [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] boot 2024-04-24T23:08:40.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:40 smithi112 ceph-mon[29186]: osdmap e33: 5 total, 5 up, 5 in 2024-04-24T23:08:40.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:40 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:41.395 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:08:41.395 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:41.395 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi150 2024-04-24T23:08:41.395 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.395 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.395 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:41.395 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:41.396 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:08:41.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:41.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi150 2024-04-24T23:08:41.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:41.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:41.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi150 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi150 to 8544M 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:41.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:41.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:41.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:41.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:41.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 -- 172.21.15.187:0/2637709316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102390 msgr2=0x7fa08c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:41.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 --2- 172.21.15.187:0/2637709316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102390 0x7fa08c1027f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009990 tx=0x7fa07c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:08:41.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 -- 172.21.15.187:0/2637709316 shutdown_connections 2024-04-24T23:08:41.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 --2- 172.21.15.187:0/2637709316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c102d30 0x7fa08c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:41.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 --2- 172.21.15.187:0/2637709316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa08c101190 0x7fa08c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:41.870 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 --2- 172.21.15.187:0/2637709316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102390 0x7fa08c1027f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 -- 172.21.15.187:0/2637709316 >> 172.21.15.187:0/2637709316 conn(0x7fa08c0fc730 msgr2=0x7fa08c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 -- 172.21.15.187:0/2637709316 shutdown_connections 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.865+0000 7fa0938ed700 1 -- 172.21.15.187:0/2637709316 wait complete. 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa0938ed700 1 Processor -- start 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa0938ed700 1 -- start start 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa0938ed700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa08c101190 0x7fa08c06aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa0938ed700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c102390 0x7fa08c19d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa0938ed700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102d30 0x7fa08c19f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa0938ed700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10ced0 con 0x7fa08c101190 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa0938ed700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10cd50 con 0x7fa08c102390 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa0938ed700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c10d050 con 0x7fa08c102d30 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091e8a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102d30 0x7fa08c19f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa090e88700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c102390 0x7fa08c19d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:41.871 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091e8a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102d30 0x7fa08c19f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:34446/0 (socket says 172.21.15.187:34446) 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091e8a700 1 -- 172.21.15.187:0/84586761 learned_addr learned my addr 172.21.15.187:0/84586761 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091689700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa08c101190 0x7fa08c06aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091e8a700 1 -- 172.21.15.187:0/84586761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c102390 msgr2=0x7fa08c19d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091e8a700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c102390 0x7fa08c19d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091e8a700 1 -- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa08c101190 msgr2=0x7fa08c06aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091e8a700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa08c101190 0x7fa08c06aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091e8a700 1 -- 172.21.15.187:0/84586761 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c1a0200 con 0x7fa08c102d30 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.866+0000 7fa091689700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa08c101190 0x7fa08c06aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.867+0000 7fa091e8a700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102d30 0x7fa08c19f930 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa08400ada0 tx=0x7fa08400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.867+0000 7fa0827fc700 1 -- 172.21.15.187:0/84586761 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084017020 con 0x7fa08c102d30 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.867+0000 7fa0827fc700 1 -- 172.21.15.187:0/84586761 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa08400a0d0 con 0x7fa08c102d30 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.867+0000 7fa0827fc700 1 -- 172.21.15.187:0/84586761 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084005490 con 0x7fa08c102d30 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.867+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c1a04f0 con 0x7fa08c102d30 2024-04-24T23:08:41.872 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.867+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c1a0a80 con 0x7fa08c102d30 2024-04-24T23:08:41.873 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.868+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c068f20 con 0x7fa08c102d30 2024-04-24T23:08:41.873 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.869+0000 7fa0827fc700 1 -- 172.21.15.187:0/84586761 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa08400c980 con 0x7fa08c102d30 2024-04-24T23:08:41.874 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.869+0000 7fa0827fc700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0780a2ca0 0x7fa0780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:41.874 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.870+0000 7fa0827fc700 1 -- 172.21.15.187:0/84586761 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fa0840cdcd0 con 0x7fa08c102d30 2024-04-24T23:08:41.874 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.870+0000 7fa091689700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0780a2ca0 0x7fa0780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:41.874 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.870+0000 7fa091689700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0780a2ca0 0x7fa0780a5160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa088005950 tx=0x7fa0880058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:41.875 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:41.873+0000 7fa0827fc700 1 -- 172.21.15.187:0/84586761 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa084021020 con 0x7fa08c102d30 2024-04-24T23:08:42.078 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:08:42.076+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa08c062670 con 0x7fa0780a2ca0 2024-04-24T23:08:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:42 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:42 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:42 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:42.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:42.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:42.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:42 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:42.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:42 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:42.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:42 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:42.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:42 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:43.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:43 smithi112 ceph-mon[29186]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:43 smithi112 ceph-mon[29186]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:43.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:43 smithi150 ceph-mon[32247]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:43.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:43 smithi150 ceph-mon[32247]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:43.620 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:43 smithi187 ceph-mon[33292]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:43.620 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:43 smithi187 ceph-mon[33292]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:45.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:44 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/4095028869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]: dispatch 2024-04-24T23:08:45.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:44 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]: dispatch 2024-04-24T23:08:45.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:44 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]': finished 2024-04-24T23:08:45.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:44 smithi112 ceph-mon[29186]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:08:45.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:44 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:08:45.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:44 smithi150 ceph-mon[32247]: from='client.? 172.21.15.187:0/4095028869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]: dispatch 2024-04-24T23:08:45.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:44 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]: dispatch 2024-04-24T23:08:45.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:44 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]': finished 2024-04-24T23:08:45.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:44 smithi150 ceph-mon[32247]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:08:45.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:44 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:08:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:44 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/4095028869' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]: dispatch 2024-04-24T23:08:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:44 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]: dispatch 2024-04-24T23:08:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:44 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "905c0bbc-34be-4035-9799-e3385ea78043"}]': finished 2024-04-24T23:08:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:44 smithi187 ceph-mon[33292]: osdmap e35: 6 total, 5 up, 6 in 2024-04-24T23:08:45.073 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:44 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:08:46.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:45 smithi112 ceph-mon[29186]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:46.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:45 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/1220815998' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:46.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:45 smithi150 ceph-mon[32247]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:46.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:45 smithi150 ceph-mon[32247]: from='client.? 172.21.15.187:0/1220815998' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:45 smithi187 ceph-mon[33292]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:45 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/1220815998' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:47 smithi112 ceph-mon[29186]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:48.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:47 smithi150 ceph-mon[32247]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:47 smithi187 ceph-mon[33292]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:49.866 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:49 smithi187 ceph-mon[33292]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:50.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:49 smithi112 ceph-mon[29186]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:50.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:49 smithi150 ceph-mon[32247]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:51.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:51 smithi187 ceph-mon[33292]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:52.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:51 smithi112 ceph-mon[29186]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:52.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:51 smithi150 ceph-mon[32247]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:52.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:52 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:08:52.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:52 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:53.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:52 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:08:53.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:52 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:53.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:52 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:08:53.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:52 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:53.956 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:53 smithi187 ceph-mon[33292]: Deploying daemon osd.5 on smithi187 2024-04-24T23:08:53.957 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:53 smithi187 ceph-mon[33292]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:54.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:53 smithi150 ceph-mon[32247]: Deploying daemon osd.5 on smithi187 2024-04-24T23:08:54.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:53 smithi150 ceph-mon[32247]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:53 smithi112 ceph-mon[29186]: Deploying daemon osd.5 on smithi187 2024-04-24T23:08:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:53 smithi112 ceph-mon[29186]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:56.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:55 smithi150 ceph-mon[32247]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:55 smithi187 ceph-mon[33292]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:55 smithi112 ceph-mon[29186]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:57 smithi150 ceph-mon[32247]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:58.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:58.148 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:57 smithi187 ceph-mon[33292]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:58.148 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:58.149 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:58.149 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:58.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:57 smithi112 ceph-mon[29186]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:08:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:08:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:00.042 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:08:59 smithi187 ceph-mon[33292]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:00.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:08:59 smithi150 ceph-mon[32247]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:00.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:08:59 smithi112 ceph-mon[29186]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:01.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:01.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:01.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:01 smithi187 ceph-mon[33292]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:01.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.244+0000 7fa0827fc700 1 -- 172.21.15.187:0/84586761 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa08c062670 con 0x7fa0780a2ca0 2024-04-24T23:09:01.246 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 5 on host 'smithi187' 2024-04-24T23:09:01.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0780a2ca0 msgr2=0x7fa0780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0780a2ca0 0x7fa0780a5160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa088005950 tx=0x7fa0880058e0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102d30 msgr2=0x7fa08c19f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102d30 0x7fa08c19f930 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa08400ada0 tx=0x7fa08400c330 comp rx=0 tx=0).stop 2024-04-24T23:09:01.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 shutdown_connections 2024-04-24T23:09:01.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0780a2ca0 0x7fa0780a5160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.248 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa08c102d30 0x7fa08c19f930 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa08c101190 0x7fa08c06aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 --2- 172.21.15.187:0/84586761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa08c102390 0x7fa08c19d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 >> 172.21.15.187:0/84586761 conn(0x7fa08c0fc730 msgr2=0x7fa08c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:01.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 shutdown_connections 2024-04-24T23:09:01.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:01.247+0000 7fa0938ed700 1 -- 172.21.15.187:0/84586761 wait complete. 2024-04-24T23:09:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:01 smithi112 ceph-mon[29186]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:01.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:01.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:01.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:01.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:01 smithi150 ceph-mon[32247]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:01.895 DEBUG:teuthology.orchestra.run.smithi187:osd.5> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T23:09:01.897 INFO:tasks.cephadm:Deploying osd.6 on smithi112 with /dev/vg_nvme/lv_2... 2024-04-24T23:09:01.897 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:09:02.144 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:09:02.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:02 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:02.493 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:02 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:02.493 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:02 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:02.493 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:02 smithi187 ceph-mon[33292]: from='osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:02.493 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:02 smithi187 ceph-mon[33292]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:02.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:02 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:02.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:02 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:02.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:02 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:02.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:02 smithi112 ceph-mon[29186]: from='osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:02.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:02 smithi112 ceph-mon[29186]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:02.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:02 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:02.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:02 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:02.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:02 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:02.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:02 smithi150 ceph-mon[32247]: from='osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:02.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:02 smithi150 ceph-mon[32247]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:03.278 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:03 smithi187 ceph-mon[33292]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:09:03.278 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:03 smithi187 ceph-mon[33292]: from='osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:09:03.279 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:03 smithi187 ceph-mon[33292]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:09:03.279 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:03.279 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:03 smithi187 ceph-mon[33292]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:09:03.279 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:03 smithi187 ceph-mon[33292]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:03 smithi112 ceph-mon[29186]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:09:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:03 smithi112 ceph-mon[29186]: from='osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:09:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:03 smithi112 ceph-mon[29186]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:09:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:03 smithi112 ceph-mon[29186]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:09:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:03 smithi112 ceph-mon[29186]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:03 smithi150 ceph-mon[32247]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:09:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:03 smithi150 ceph-mon[32247]: from='osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:09:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:03 smithi150 ceph-mon[32247]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:09:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:03 smithi150 ceph-mon[32247]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:09:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:03 smithi150 ceph-mon[32247]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:04.118 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:09:04.119 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:09:04.119 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-24T23:09:04.119 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-24T23:09:04.119 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142354 s, 737 MB/s 2024-04-24T23:09:04.119 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-24T23:09:04.237 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:09:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:09:03.950+0000 7f6a6a705700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:09:04.237 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:09:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:09:03.958+0000 7f6a624f0700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:09:04.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:04 smithi112 ceph-mon[29186]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:09:04.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:04 smithi112 ceph-mon[29186]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:09:04.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:04 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:04 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:04 smithi112 ceph-mon[29186]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:09:04.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:04 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:04 smithi112 ceph-mon[29186]: from='osd.5 ' entity='osd.5' 2024-04-24T23:09:04.518 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:04 smithi187 ceph-mon[33292]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:09:04.518 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:04 smithi187 ceph-mon[33292]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:09:04.518 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:04 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.518 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:04 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.518 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:04 smithi187 ceph-mon[33292]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:09:04.518 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:04 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.518 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:04 smithi187 ceph-mon[33292]: from='osd.5 ' entity='osd.5' 2024-04-24T23:09:04.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:04 smithi150 ceph-mon[32247]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:09:04.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:04 smithi150 ceph-mon[32247]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:09:04.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:04 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:04 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:04 smithi150 ceph-mon[32247]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:09:04.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:04 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:04 smithi150 ceph-mon[32247]: from='osd.5 ' entity='osd.5' 2024-04-24T23:09:04.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-04-24T23:09:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:09:05.422 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:09:05.422 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:09:05.422 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] boot 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi187 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi187 to 8544M 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:05.423 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] boot 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi187 2024-04-24T23:09:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi187 to 8544M 2024-04-24T23:09:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:09:05.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:09:05.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:05.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-24T23:09:05.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: osd.5 [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] boot 2024-04-24T23:09:05.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:09:05.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:05.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi187 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi187 to 8544M 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:05.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.571+0000 7fd200712700 1 -- 172.21.15.112:0/3228447123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103490 msgr2=0x7fd1f81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.571+0000 7fd200712700 1 --2- 172.21.15.112:0/3228447123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103490 0x7fd1f81038f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80099c0 tx=0x7fd1e8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.571+0000 7fd200712700 1 -- 172.21.15.112:0/3228447123 shutdown_connections 2024-04-24T23:09:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.571+0000 7fd200712700 1 --2- 172.21.15.112:0/3228447123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1f8102290 0x7fd1f81026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.571+0000 7fd200712700 1 --2- 172.21.15.112:0/3228447123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1f8103e30 0x7fd1f810a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.571+0000 7fd200712700 1 --2- 172.21.15.112:0/3228447123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103490 0x7fd1f81038f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.571+0000 7fd200712700 1 -- 172.21.15.112:0/3228447123 >> 172.21.15.112:0/3228447123 conn(0x7fd1f80fd830 msgr2=0x7fd1f80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 -- 172.21.15.112:0/3228447123 shutdown_connections 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 -- 172.21.15.112:0/3228447123 wait complete. 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 Processor -- start 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 -- start start 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1f8102290 0x7fd1f81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1f8103490 0x7fd1f8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103e30 0x7fd1f819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f810e030 con 0x7fd1f8103e30 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f810deb0 con 0x7fd1f8103490 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd200712700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f810e1b0 con 0x7fd1f8102290 2024-04-24T23:09:06.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd1fecaf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103e30 0x7fd1f819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd1fecaf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103e30 0x7fd1f819fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32914/0 (socket says 172.21.15.112:32914) 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd1fecaf700 1 -- 172.21.15.112:0/2641325948 learned_addr learned my addr 172.21.15.112:0/2641325948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.572+0000 7fd1fe4ae700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1f8102290 0x7fd1f81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fdcad700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1f8103490 0x7fd1f8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fecaf700 1 -- 172.21.15.112:0/2641325948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1f8102290 msgr2=0x7fd1f81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fecaf700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1f8102290 0x7fd1f81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fecaf700 1 -- 172.21.15.112:0/2641325948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1f8103490 msgr2=0x7fd1f8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fecaf700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1f8103490 0x7fd1f8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fecaf700 1 -- 172.21.15.112:0/2641325948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f81a0220 con 0x7fd1f8103e30 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fe4ae700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1f8102290 0x7fd1f81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:06.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fdcad700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1f8103490 0x7fd1f8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1fecaf700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103e30 0x7fd1f819fb80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0007da0 tx=0x7fd1f000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1ef7fe700 1 -- 172.21.15.112:0/2641325948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0016020 con 0x7fd1f8103e30 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1ef7fe700 1 -- 172.21.15.112:0/2641325948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd1f000ad00 con 0x7fd1f8103e30 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f81a0510 con 0x7fd1f8103e30 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd1ef7fe700 1 -- 172.21.15.112:0/2641325948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0011820 con 0x7fd1f8103e30 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.573+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f81a0a50 con 0x7fd1f8103e30 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.574+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f8068f20 con 0x7fd1f8103e30 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.575+0000 7fd1ef7fe700 1 -- 172.21.15.112:0/2641325948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd1f0019020 con 0x7fd1f8103e30 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.575+0000 7fd1ef7fe700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd1e40a2c20 0x7fd1e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.576+0000 7fd1fe4ae700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd1e40a2c20 0x7fd1e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.576+0000 7fd1ef7fe700 1 -- 172.21.15.112:0/2641325948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fd1f00ce5f0 con 0x7fd1f8103e30 2024-04-24T23:09:06.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.576+0000 7fd1fe4ae700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd1e40a2c20 0x7fd1e40a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd1f81032f0 tx=0x7fd1f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:06.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.580+0000 7fd1ef7fe700 1 -- 172.21.15.112:0/2641325948 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd1f0020020 con 0x7fd1f8103e30 2024-04-24T23:09:06.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:06.789+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd1f8062670 con 0x7fd1e40a2c20 2024-04-24T23:09:07.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:06 smithi150 ceph-mon[32247]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:09:07.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:06 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:07.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:06 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:07.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:06 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:07.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:06 smithi187 ceph-mon[33292]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:09:07.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:06 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:07.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:06 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:07.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:06 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:07.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:06 smithi112 ceph-mon[29186]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:09:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:06 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:06 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:06 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:08.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:07 smithi112 ceph-mon[29186]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:08.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:07 smithi112 ceph-mon[29186]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:08.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:07 smithi112 ceph-mon[29186]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:09:08.137 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:07 smithi187 ceph-mon[33292]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:08.138 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:07 smithi187 ceph-mon[33292]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:08.138 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:07 smithi187 ceph-mon[33292]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:09:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:07 smithi150 ceph-mon[32247]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:07 smithi150 ceph-mon[32247]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:07 smithi150 ceph-mon[32247]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:09:09.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:08 smithi112 ceph-mon[29186]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:09:09.155 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:08 smithi187 ceph-mon[33292]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:09:09.218 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:08 smithi150 ceph-mon[32247]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:09:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:09 smithi187 ceph-mon[33292]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-24T23:09:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:09 smithi187 ceph-mon[33292]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-24T23:09:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:09 smithi187 ceph-mon[33292]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:09:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:09 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/392145354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b8b9f0-b530-4914-a7c6-d3b8b9fa6002"}]: dispatch 2024-04-24T23:09:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:09 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/392145354' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b8b9f0-b530-4914-a7c6-d3b8b9fa6002"}]': finished 2024-04-24T23:09:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:09 smithi187 ceph-mon[33292]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:09:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:09 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:09 smithi112 ceph-mon[29186]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-24T23:09:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:09 smithi112 ceph-mon[29186]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-24T23:09:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:09 smithi112 ceph-mon[29186]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:09:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:09 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/392145354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b8b9f0-b530-4914-a7c6-d3b8b9fa6002"}]: dispatch 2024-04-24T23:09:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:09 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/392145354' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b8b9f0-b530-4914-a7c6-d3b8b9fa6002"}]': finished 2024-04-24T23:09:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:09 smithi112 ceph-mon[29186]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:09:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:09 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:09 smithi150 ceph-mon[32247]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-24T23:09:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:09 smithi150 ceph-mon[32247]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-24T23:09:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:09 smithi150 ceph-mon[32247]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:09:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:09 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/392145354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14b8b9f0-b530-4914-a7c6-d3b8b9fa6002"}]: dispatch 2024-04-24T23:09:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:09 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/392145354' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14b8b9f0-b530-4914-a7c6-d3b8b9fa6002"}]': finished 2024-04-24T23:09:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:09 smithi150 ceph-mon[32247]: osdmap e44: 7 total, 6 up, 7 in 2024-04-24T23:09:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:09 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:11.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:10 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/750322415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:11.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:10 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/750322415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:11.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:10 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/750322415' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:12.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:11 smithi187 ceph-mon[33292]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-24T23:09:12.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:11 smithi112 ceph-mon[29186]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-24T23:09:12.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:11 smithi150 ceph-mon[32247]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-24T23:09:13.173 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:12 smithi112 ceph-mon[29186]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-24T23:09:13.173 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:12 smithi112 ceph-mon[29186]: Cluster is now healthy 2024-04-24T23:09:13.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:12 smithi187 ceph-mon[33292]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-24T23:09:13.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:12 smithi187 ceph-mon[33292]: Cluster is now healthy 2024-04-24T23:09:13.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:12 smithi150 ceph-mon[32247]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-24T23:09:13.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:12 smithi150 ceph-mon[32247]: Cluster is now healthy 2024-04-24T23:09:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:13 smithi187 ceph-mon[33292]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:09:14.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:13 smithi112 ceph-mon[29186]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:09:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:13 smithi150 ceph-mon[32247]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:09:15.999 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:15 smithi112 ceph-mon[29186]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:09:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:15 smithi187 ceph-mon[33292]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:09:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:15 smithi150 ceph-mon[32247]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:09:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:17 smithi187 ceph-mon[33292]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:09:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:17 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:09:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:17 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:17 smithi187 ceph-mon[33292]: Deploying daemon osd.6 on smithi112 2024-04-24T23:09:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:17 smithi112 ceph-mon[29186]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:09:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:17 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:09:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:17 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:17 smithi112 ceph-mon[29186]: Deploying daemon osd.6 on smithi112 2024-04-24T23:09:18.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:17 smithi150 ceph-mon[32247]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:09:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:17 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-24T23:09:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:17 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:17 smithi150 ceph-mon[32247]: Deploying daemon osd.6 on smithi112 2024-04-24T23:09:20.175 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:19 smithi112 ceph-mon[29186]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:19 smithi187 ceph-mon[33292]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:19 smithi150 ceph-mon[32247]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:09:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:21 smithi187 ceph-mon[33292]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-24T23:09:22.262 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:21 smithi112 ceph-mon[29186]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-24T23:09:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:21 smithi150 ceph-mon[32247]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-04-24T23:09:23.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:23.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:23.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:23.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:23 smithi187 ceph-mon[33292]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:09:24.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:24.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:24.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:24.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:23 smithi112 ceph-mon[29186]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:09:24.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:24.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:24.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:24.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:23 smithi150 ceph-mon[32247]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-24T23:09:25.958 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:25 smithi112 ceph-mon[29186]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:25.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:25.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:25.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:25.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:25.959 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:26.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:25 smithi150 ceph-mon[32247]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:26.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:26.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:26.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:26.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:26.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:25 smithi187 ceph-mon[33292]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:26.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.750+0000 7fd1ef7fe700 1 -- 172.21.15.112:0/2641325948 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd1f8062670 con 0x7fd1e40a2c20 2024-04-24T23:09:26.753 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 6 on host 'smithi112' 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd1e40a2c20 msgr2=0x7fd1e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd1e40a2c20 0x7fd1e40a50e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd1f81032f0 tx=0x7fd1f4008040 comp rx=0 tx=0).stop 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103e30 msgr2=0x7fd1f819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103e30 0x7fd1f819fb80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0007da0 tx=0x7fd1f000a340 comp rx=0 tx=0).stop 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 shutdown_connections 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd1e40a2c20 0x7fd1e40a50e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd1f8103e30 0x7fd1f819fb80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1f8102290 0x7fd1f81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:26.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 --2- 172.21.15.112:0/2641325948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd1f8103490 0x7fd1f8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:26.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.752+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 >> 172.21.15.112:0/2641325948 conn(0x7fd1f80fd830 msgr2=0x7fd1f8106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:26.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.753+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 shutdown_connections 2024-04-24T23:09:26.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:09:26.753+0000 7fd200712700 1 -- 172.21.15.112:0/2641325948 wait complete. 2024-04-24T23:09:27.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:26 smithi112 ceph-mon[29186]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:09:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:27.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:26 smithi150 ceph-mon[32247]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:09:27.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:27.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:27.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:26 smithi187 ceph-mon[33292]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-24T23:09:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:27.473 DEBUG:teuthology.orchestra.run.smithi112:osd.6> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.6.service 2024-04-24T23:09:27.475 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_2... 2024-04-24T23:09:27.475 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:09:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:09:28.017 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:27 smithi112 ceph-mon[29186]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:28.017 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:27 smithi112 ceph-mon[29186]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:09:28.017 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:27 smithi112 ceph-mon[29186]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:09:28.017 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:27 smithi112 ceph-mon[29186]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:09:28.017 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:28.018 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:09:27 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:09:27.810+0000 7fbd04f8f700 -1 osd.6 0 waiting for initial osdmap 2024-04-24T23:09:28.018 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:09:27 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:09:27.821+0000 7fbd00d82700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:09:28.060 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:27 smithi150 ceph-mon[32247]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:28.060 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:27 smithi150 ceph-mon[32247]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:09:28.060 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:27 smithi150 ceph-mon[32247]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:09:28.060 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:27 smithi150 ceph-mon[32247]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:09:28.061 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:27 smithi187 ceph-mon[33292]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:27 smithi187 ceph-mon[33292]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-24T23:09:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:27 smithi187 ceph-mon[33292]: osdmap e45: 7 total, 6 up, 7 in 2024-04-24T23:09:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:27 smithi187 ceph-mon[33292]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:09:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:28.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:28 smithi112 ceph-mon[29186]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:09:28.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:28 smithi112 ceph-mon[29186]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:09:28.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:28.973 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:29.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:28 smithi150 ceph-mon[32247]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:09:29.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:28 smithi150 ceph-mon[32247]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:09:29.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:29.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:28 smithi187 ceph-mon[33292]: from='osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:09:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:28 smithi187 ceph-mon[33292]: osdmap e46: 7 total, 6 up, 7 in 2024-04-24T23:09:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:29.963 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:09:29.963 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:09:29.963 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-24T23:09:29.963 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-24T23:09:29.963 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130101 s, 806 MB/s 2024-04-24T23:09:29.963 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-24T23:09:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:29 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:09:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:29 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:09:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:29 smithi150 ceph-mon[32247]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:29 smithi150 ceph-mon[32247]: osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] boot 2024-04-24T23:09:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:29 smithi150 ceph-mon[32247]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:09:30.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:29 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:30.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:29 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:09:30.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:29 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:09:30.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:29 smithi112 ceph-mon[29186]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:30.179 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:29 smithi112 ceph-mon[29186]: osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] boot 2024-04-24T23:09:30.180 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:29 smithi112 ceph-mon[29186]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:09:30.180 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:29 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:29 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:09:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:29 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:09:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:29 smithi187 ceph-mon[33292]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:09:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:29 smithi187 ceph-mon[33292]: osd.6 [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] boot 2024-04-24T23:09:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:29 smithi187 ceph-mon[33292]: osdmap e47: 7 total, 7 up, 7 in 2024-04-24T23:09:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:29 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-24T23:09:30.644 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-04-24T23:09:30.895 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:30.933 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:30 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:31.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:30 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: osdmap e48: 7 total, 7 up, 7 in 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:31.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:30 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:32.061 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:31 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi112 2024-04-24T23:09:32.061 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:31 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-24T23:09:32.062 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:31 smithi150 ceph-mon[32247]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:31 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi112 2024-04-24T23:09:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:31 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-24T23:09:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:31 smithi187 ceph-mon[33292]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:31 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi112 2024-04-24T23:09:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:31 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-24T23:09:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:31 smithi112 ceph-mon[29186]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 -- 172.21.15.150:0/3778320961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 msgr2=0x7f3218102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 --2- 172.21.15.150:0/3778320961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 0x7f3218102700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f32080099c0 tx=0x7f3208009cd0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 -- 172.21.15.150:0/3778320961 shutdown_connections 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 --2- 172.21.15.150:0/3778320961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 0x7f3218102700 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 --2- 172.21.15.150:0/3778320961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3218103e80 0x7f321810a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 --2- 172.21.15.150:0/3778320961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32181034e0 0x7f3218103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 -- 172.21.15.150:0/3778320961 >> 172.21.15.150:0/3778320961 conn(0x7f32180fd880 msgr2=0x7f32180ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 -- 172.21.15.150:0/3778320961 shutdown_connections 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.578+0000 7f321d85c700 1 -- 172.21.15.150:0/3778320961 wait complete. 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.579+0000 7f321d85c700 1 Processor -- start 2024-04-24T23:09:32.593 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.579+0000 7f321d85c700 1 -- start start 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.579+0000 7f321d85c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 0x7f3218198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.579+0000 7f321d85c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32181034e0 0x7f3218198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.579+0000 7f321d85c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3218103e80 0x7f321819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.579+0000 7f321d85c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f321810e020 con 0x7f32181034e0 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.579+0000 7f321d85c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f321810dea0 con 0x7f32181022e0 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.579+0000 7f321d85c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f321810e1a0 con 0x7f3218103e80 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 0x7f3218198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 0x7f3218198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:59568/0 (socket says 172.21.15.150:59568) 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 -- 172.21.15.150:0/3920954269 learned_addr learned my addr 172.21.15.150:0/3920954269 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f32177fe700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3218103e80 0x7f321819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f32167fc700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32181034e0 0x7f3218198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.594 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 -- 172.21.15.150:0/3920954269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3218103e80 msgr2=0x7f321819fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3218103e80 0x7f321819fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 -- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32181034e0 msgr2=0x7f3218198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32181034e0 0x7f3218198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 -- 172.21.15.150:0/3920954269 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32181a01e0 con 0x7f32181022e0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f32177fe700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3218103e80 0x7f321819fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f3216ffd700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 0x7f3218198580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3208003990 tx=0x7f3208003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f321c85a700 1 -- 172.21.15.150:0/3920954269 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208027070 con 0x7f32181022e0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f321c85a700 1 -- 172.21.15.150:0/3920954269 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f320800bda0 con 0x7f32181022e0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f321c85a700 1 -- 172.21.15.150:0/3920954269 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320800f6c0 con 0x7f32181022e0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32181a0470 con 0x7f32181022e0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.580+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32181a0950 con 0x7f32181022e0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.581+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3218068f20 con 0x7f32181022e0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.582+0000 7f321c85a700 1 -- 172.21.15.150:0/3920954269 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f320802a070 con 0x7f32181022e0 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.583+0000 7f321c85a700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32000a2c20 0x7f32000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:32.595 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.583+0000 7f321c85a700 1 -- 172.21.15.150:0/3920954269 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f32080cf760 con 0x7f32181022e0 2024-04-24T23:09:32.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.583+0000 7f32167fc700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32000a2c20 0x7f32000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:32.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.584+0000 7f32167fc700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32000a2c20 0x7f32000a50e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f320c005950 tx=0x7f320c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:32.596 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.586+0000 7f321c85a700 1 -- 172.21.15.150:0/3920954269 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3208031070 con 0x7f32181022e0 2024-04-24T23:09:32.795 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:32.793+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3218062670 con 0x7f32000a2c20 2024-04-24T23:09:33.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:32 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:33.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:32 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:33.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:32 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:33.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:32 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:33.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:32 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:33.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:32 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:33.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:32 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:33.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:32 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:33.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:32 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:34.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:33 smithi150 ceph-mon[32247]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:34.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:33 smithi150 ceph-mon[32247]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:33 smithi187 ceph-mon[33292]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:33 smithi187 ceph-mon[33292]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:34.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:33 smithi112 ceph-mon[29186]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:34.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:33 smithi112 ceph-mon[29186]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:35 smithi187 ceph-mon[33292]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:35 smithi187 ceph-mon[33292]: from='client.? 172.21.15.150:0/1404880777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]: dispatch 2024-04-24T23:09:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:35 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]: dispatch 2024-04-24T23:09:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:35 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]': finished 2024-04-24T23:09:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:35 smithi187 ceph-mon[33292]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:09:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:36.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:35 smithi112 ceph-mon[29186]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:36.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:35 smithi112 ceph-mon[29186]: from='client.? 172.21.15.150:0/1404880777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]: dispatch 2024-04-24T23:09:36.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:35 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]: dispatch 2024-04-24T23:09:36.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:35 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]': finished 2024-04-24T23:09:36.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:35 smithi112 ceph-mon[29186]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:09:36.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:35 smithi150 ceph-mon[32247]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:35 smithi150 ceph-mon[32247]: from='client.? 172.21.15.150:0/1404880777' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]: dispatch 2024-04-24T23:09:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:35 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]: dispatch 2024-04-24T23:09:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:35 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aacba960-f7c4-41cd-8ee0-cac0fba62c5d"}]': finished 2024-04-24T23:09:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:35 smithi150 ceph-mon[32247]: osdmap e49: 8 total, 7 up, 8 in 2024-04-24T23:09:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:37.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:36 smithi187 ceph-mon[33292]: from='client.? 172.21.15.150:0/3371094711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:37.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:36 smithi112 ceph-mon[29186]: from='client.? 172.21.15.150:0/3371094711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:37.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:36 smithi150 ceph-mon[32247]: from='client.? 172.21.15.150:0/3371094711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:38.137 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:37 smithi187 ceph-mon[33292]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:38.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:37 smithi112 ceph-mon[29186]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:37 smithi150 ceph-mon[32247]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:40.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:39 smithi187 ceph-mon[33292]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:40.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:39 smithi112 ceph-mon[29186]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:40.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:39 smithi150 ceph-mon[32247]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:41.965 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:41 smithi150 ceph-mon[32247]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:41 smithi187 ceph-mon[33292]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:42.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:41 smithi112 ceph-mon[29186]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:43.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:43 smithi150 ceph-mon[32247]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:43.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:43 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:09:43.996 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:43 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:43 smithi187 ceph-mon[33292]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:43 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:09:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:43 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:43 smithi112 ceph-mon[29186]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:43 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-24T23:09:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:43 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:45.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:44 smithi150 ceph-mon[32247]: Deploying daemon osd.7 on smithi150 2024-04-24T23:09:45.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:44 smithi187 ceph-mon[33292]: Deploying daemon osd.7 on smithi150 2024-04-24T23:09:45.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:44 smithi112 ceph-mon[29186]: Deploying daemon osd.7 on smithi150 2024-04-24T23:09:46.166 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:45 smithi150 ceph-mon[32247]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:45 smithi187 ceph-mon[33292]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:46.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:45 smithi112 ceph-mon[29186]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:48.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:47 smithi150 ceph-mon[32247]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:47 smithi187 ceph-mon[33292]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:48.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:47 smithi112 ceph-mon[29186]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:49 smithi187 ceph-mon[33292]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:49 smithi112 ceph-mon[29186]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:50.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:50.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:49 smithi150 ceph-mon[32247]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:52.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:51 smithi150 ceph-mon[32247]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:52.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:51 smithi187 ceph-mon[33292]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:51 smithi112 ceph-mon[29186]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:52.611 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.609+0000 7f321c85a700 1 -- 172.21.15.150:0/3920954269 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3218062670 con 0x7f32000a2c20 2024-04-24T23:09:52.611 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32000a2c20 msgr2=0x7f32000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32000a2c20 0x7f32000a50e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f320c005950 tx=0x7f320c0058e0 comp rx=0 tx=0).stop 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 msgr2=0x7f3218198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 0x7f3218198580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3208003990 tx=0x7f3208003d10 comp rx=0 tx=0).stop 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 shutdown_connections 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32000a2c20 0x7f32000a50e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32181022e0 0x7f3218198580 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3218103e80 0x7f321819fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 --2- 172.21.15.150:0/3920954269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32181034e0 0x7f3218198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 >> 172.21.15.150:0/3920954269 conn(0x7f32180fd880 msgr2=0x7f32181067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 shutdown_connections 2024-04-24T23:09:52.614 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:09:52.612+0000 7f321d85c700 1 -- 172.21.15.150:0/3920954269 wait complete. 2024-04-24T23:09:53.263 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.7.service 2024-04-24T23:09:53.265 INFO:tasks.cephadm:Deploying osd.8 on smithi187 with /dev/vg_nvme/lv_2... 2024-04-24T23:09:53.266 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:09:53.515 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:09:53.868 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:53.868 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:53.869 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:53 smithi187 ceph-mon[33292]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:53.869 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:53 smithi187 ceph-mon[33292]: from='osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:09:53.869 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:53 smithi187 ceph-mon[33292]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:09:53.869 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:53.869 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:53.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:54.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:54.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:53 smithi150 ceph-mon[32247]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:54.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:53 smithi150 ceph-mon[32247]: from='osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:09:54.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:53 smithi150 ceph-mon[32247]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:09:54.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:54.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:54.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:54.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:54.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:53 smithi112 ceph-mon[29186]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:54.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:53 smithi112 ceph-mon[29186]: from='osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:09:54.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:53 smithi112 ceph-mon[29186]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-24T23:09:54.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:54.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:54.878 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:54 smithi187 ceph-mon[33292]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:09:54.879 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:54 smithi187 ceph-mon[33292]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:09:54.879 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:54 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:54.879 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:54 smithi187 ceph-mon[33292]: from='osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:09:54.879 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:54 smithi187 ceph-mon[33292]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:09:55.024 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:54 smithi150 ceph-mon[32247]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:09:55.024 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:54 smithi150 ceph-mon[32247]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:09:55.024 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:54 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:55.024 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:54 smithi150 ceph-mon[32247]: from='osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:09:55.024 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:54 smithi150 ceph-mon[32247]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:09:55.024 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:09:54 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:09:54.604+0000 7f1fa4a52700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T23:09:55.025 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:09:54 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:09:54.621+0000 7f1fa0845700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:09:55.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:54 smithi112 ceph-mon[29186]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-24T23:09:55.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:54 smithi112 ceph-mon[29186]: osdmap e50: 8 total, 7 up, 8 in 2024-04-24T23:09:55.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:54 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:54 smithi112 ceph-mon[29186]: from='osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:09:55.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:54 smithi112 ceph-mon[29186]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:09:55.612 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:09:55.612 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:09:55.612 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2024-04-24T23:09:55.612 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2024-04-24T23:09:55.612 INFO:teuthology.orchestra.run.smithi187.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115789 s, 906 MB/s 2024-04-24T23:09:55.612 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2024-04-24T23:09:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:55 smithi187 ceph-mon[33292]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:09:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:55 smithi187 ceph-mon[33292]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:09:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:55.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:55.737 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:55 smithi187 ceph-mon[33292]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:55 smithi112 ceph-mon[29186]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:09:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:55 smithi112 ceph-mon[29186]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:09:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:56.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:55 smithi112 ceph-mon[29186]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:56.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:55 smithi150 ceph-mon[32247]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:09:56.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:55 smithi150 ceph-mon[32247]: osdmap e51: 8 total, 7 up, 8 in 2024-04-24T23:09:56.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:56.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:56.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:55 smithi150 ceph-mon[32247]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-24T23:09:56.332 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi187:vg_nvme/lv_2 2024-04-24T23:09:56.577 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:09:56.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:56 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:09:56.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:56 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:09:56.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:56 smithi150 ceph-mon[32247]: osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] boot 2024-04-24T23:09:56.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:56 smithi150 ceph-mon[32247]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:09:56.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:56 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:56.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:56 smithi150 ceph-mon[32247]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:09:56.934 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:56 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:09:56.934 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:56 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:09:56.934 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:56 smithi187 ceph-mon[33292]: osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] boot 2024-04-24T23:09:56.934 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:56 smithi187 ceph-mon[33292]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:09:56.935 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:56 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:56.935 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:56 smithi187 ceph-mon[33292]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:09:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:56 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:09:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:56 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:09:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:56 smithi112 ceph-mon[29186]: osd.7 [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] boot 2024-04-24T23:09:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:56 smithi112 ceph-mon[29186]: osdmap e52: 8 total, 8 up, 8 in 2024-04-24T23:09:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:56 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-24T23:09:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:56 smithi112 ceph-mon[29186]: osdmap e53: 8 total, 8 up, 8 in 2024-04-24T23:09:58.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 -- 172.21.15.187:0/103874869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103e30 msgr2=0x7f726010a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:58.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 --2- 172.21.15.187:0/103874869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103e30 0x7f726010a6e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f725800b0a0 tx=0x7f725800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 -- 172.21.15.187:0/103874869 shutdown_connections 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 --2- 172.21.15.187:0/103874869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103e30 0x7f726010a6e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 --2- 172.21.15.187:0/103874869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7260102290 0x7f72601026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 --2- 172.21.15.187:0/103874869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260103490 0x7f72601038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 -- 172.21.15.187:0/103874869 >> 172.21.15.187:0/103874869 conn(0x7f72600fd830 msgr2=0x7f72600ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 -- 172.21.15.187:0/103874869 shutdown_connections 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.216+0000 7f7267e7e700 1 -- 172.21.15.187:0/103874869 wait complete. 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7267e7e700 1 Processor -- start 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7267e7e700 1 -- start start 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7267e7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7260102290 0x7f7260198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7267e7e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103490 0x7f7260198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7267e7e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260103e30 0x7f726019fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7267e7e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f726010e010 con 0x7f7260102290 2024-04-24T23:09:58.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7267e7e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f726010de90 con 0x7f7260103e30 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7267e7e700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f726010e190 con 0x7f7260103490 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265419700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103490 0x7f7260198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265419700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103490 0x7f7260198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:50322/0 (socket says 172.21.15.187:50322) 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265419700 1 -- 172.21.15.187:0/698103847 learned_addr learned my addr 172.21.15.187:0/698103847 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f726641b700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260103e30 0x7f726019fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265c1a700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7260102290 0x7f7260198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265419700 1 -- 172.21.15.187:0/698103847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260103e30 msgr2=0x7f726019fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265419700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260103e30 0x7f726019fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265419700 1 -- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7260102290 msgr2=0x7f7260198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265419700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7260102290 0x7f7260198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.217+0000 7f7265419700 1 -- 172.21.15.187:0/698103847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72601a0160 con 0x7f7260103490 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.218+0000 7f7265419700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103490 0x7f7260198a40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f725000d9b0 tx=0x7f725000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:58.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.218+0000 7f7256ffd700 1 -- 172.21.15.187:0/698103847 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250016070 con 0x7f7260103490 2024-04-24T23:09:58.223 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.218+0000 7f7256ffd700 1 -- 172.21.15.187:0/698103847 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7250004d30 con 0x7f7260103490 2024-04-24T23:09:58.223 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.218+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72601a0450 con 0x7f7260103490 2024-04-24T23:09:58.223 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.218+0000 7f7256ffd700 1 -- 172.21.15.187:0/698103847 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250005580 con 0x7f7260103490 2024-04-24T23:09:58.223 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.218+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72601a0990 con 0x7f7260103490 2024-04-24T23:09:58.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.219+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7260068f20 con 0x7f7260103490 2024-04-24T23:09:58.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.220+0000 7f7256ffd700 1 -- 172.21.15.187:0/698103847 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7250004220 con 0x7f7260103490 2024-04-24T23:09:58.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.220+0000 7f7256ffd700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f724c0a2c20 0x7f724c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:58.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.221+0000 7f7256ffd700 1 -- 172.21.15.187:0/698103847 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f72500ce740 con 0x7f7260103490 2024-04-24T23:09:58.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.221+0000 7f7265c1a700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f724c0a2c20 0x7f724c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:58.224 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.221+0000 7f7265c1a700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f724c0a2c20 0x7f724c0a50e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72601032f0 tx=0x7f725c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:58.226 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.224+0000 7f7256ffd700 1 -- 172.21.15.187:0/698103847 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f725001f020 con 0x7f7260103490 2024-04-24T23:09:58.226 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:09:58.226 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi150 2024-04-24T23:09:58.226 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:58.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.228 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:57 smithi187 ceph-mon[33292]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:09:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:09:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi150 2024-04-24T23:09:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:57 smithi112 ceph-mon[29186]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:09:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:09:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi150 2024-04-24T23:09:58.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:58.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi150 to 5696M 2024-04-24T23:09:58.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:58.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:58.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:58.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:58.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:58.321 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:09:58.321 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:57 smithi150 ceph-mon[32247]: osdmap e54: 8 total, 8 up, 8 in 2024-04-24T23:09:58.431 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:09:58.429+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7260062670 con 0x7f724c0a2c20 2024-04-24T23:09:59.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:59.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:59.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:59.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:58 smithi112 ceph-mon[29186]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:09:59.229 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:59.229 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:59.229 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:59.229 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:58 smithi187 ceph-mon[33292]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:09:59.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:59.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:59.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:59.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:58 smithi150 ceph-mon[32247]: osdmap e55: 8 total, 8 up, 8 in 2024-04-24T23:10:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:59 smithi187 ceph-mon[33292]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:59 smithi187 ceph-mon[33292]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:09:59 smithi187 ceph-mon[33292]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:10:00.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:59 smithi112 ceph-mon[29186]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:00.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:59 smithi112 ceph-mon[29186]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:00.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:09:59 smithi112 ceph-mon[29186]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:10:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:59 smithi150 ceph-mon[32247]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:59 smithi150 ceph-mon[32247]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:09:59 smithi150 ceph-mon[32247]: osdmap e56: 8 total, 8 up, 8 in 2024-04-24T23:10:01.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:00 smithi187 ceph-mon[33292]: overall HEALTH_OK 2024-04-24T23:10:01.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:00 smithi112 ceph-mon[29186]: overall HEALTH_OK 2024-04-24T23:10:01.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:00 smithi150 ceph-mon[32247]: overall HEALTH_OK 2024-04-24T23:10:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:01 smithi187 ceph-mon[33292]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:01 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/3744054096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]: dispatch 2024-04-24T23:10:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:01 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]: dispatch 2024-04-24T23:10:02.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:01 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]': finished 2024-04-24T23:10:02.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:01 smithi187 ceph-mon[33292]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:10:02.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:02.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:01 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/1663164127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:01 smithi112 ceph-mon[29186]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:01 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/3744054096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]: dispatch 2024-04-24T23:10:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:01 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]: dispatch 2024-04-24T23:10:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:01 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]': finished 2024-04-24T23:10:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:01 smithi112 ceph-mon[29186]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:10:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:01 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/1663164127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:02.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:01 smithi150 ceph-mon[32247]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:01 smithi150 ceph-mon[32247]: from='client.? 172.21.15.187:0/3744054096' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]: dispatch 2024-04-24T23:10:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:01 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]: dispatch 2024-04-24T23:10:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:01 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c907152a-2f78-4da1-8c5a-853706d3c59f"}]': finished 2024-04-24T23:10:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:01 smithi150 ceph-mon[32247]: osdmap e57: 9 total, 8 up, 9 in 2024-04-24T23:10:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:01 smithi150 ceph-mon[32247]: from='client.? 172.21.15.187:0/1663164127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:04.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:03 smithi112 ceph-mon[29186]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:04.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:03 smithi150 ceph-mon[32247]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:04.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:03 smithi187 ceph-mon[33292]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:06.117 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:05 smithi187 ceph-mon[33292]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:06.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:05 smithi112 ceph-mon[29186]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:06.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:05 smithi150 ceph-mon[32247]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:08.130 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:07 smithi187 ceph-mon[33292]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:08.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:07 smithi112 ceph-mon[29186]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:08.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:07 smithi150 ceph-mon[32247]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:09.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:08 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:10:09.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:08 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:09.312 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:08 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:10:09.312 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:08 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:09.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:08 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-24T23:10:09.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:08 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:10.273 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:09 smithi187 ceph-mon[33292]: Deploying daemon osd.8 on smithi187 2024-04-24T23:10:10.273 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:09 smithi187 ceph-mon[33292]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:10.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:09 smithi112 ceph-mon[29186]: Deploying daemon osd.8 on smithi187 2024-04-24T23:10:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:09 smithi112 ceph-mon[29186]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:09 smithi150 ceph-mon[32247]: Deploying daemon osd.8 on smithi187 2024-04-24T23:10:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:09 smithi150 ceph-mon[32247]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:12.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:12 smithi112 ceph-mon[29186]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:12.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:12 smithi150 ceph-mon[32247]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:12.347 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:12 smithi187 ceph-mon[33292]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:14 smithi187 ceph-mon[33292]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:14.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:14 smithi112 ceph-mon[29186]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:14.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:14 smithi150 ceph-mon[32247]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:15 smithi187 ceph-mon[33292]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:15.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:15.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:15.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:15.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:15 smithi150 ceph-mon[32247]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:15.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:15.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:15.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:15.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:15 smithi112 ceph-mon[29186]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:17.345 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.344+0000 7f7256ffd700 1 -- 172.21.15.187:0/698103847 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7260062670 con 0x7f724c0a2c20 2024-04-24T23:10:17.345 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 8 on host 'smithi187' 2024-04-24T23:10:17.347 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f724c0a2c20 msgr2=0x7f724c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.347 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f724c0a2c20 0x7f724c0a50e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72601032f0 tx=0x7f725c007400 comp rx=0 tx=0).stop 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103490 msgr2=0x7f7260198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103490 0x7f7260198a40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f725000d9b0 tx=0x7f725000dcc0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 shutdown_connections 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f724c0a2c20 0x7f724c0a50e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7260103e30 0x7f726019fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7260102290 0x7f7260198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 --2- 172.21.15.187:0/698103847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7260103490 0x7f7260198a40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 >> 172.21.15.187:0/698103847 conn(0x7f72600fd830 msgr2=0x7f7260106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 shutdown_connections 2024-04-24T23:10:17.348 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:10:17.346+0000 7f7267e7e700 1 -- 172.21.15.187:0/698103847 wait complete. 2024-04-24T23:10:17.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:17 smithi187 ceph-mon[33292]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:17.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:17 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:17.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:17 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:18.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:17 smithi150 ceph-mon[32247]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:18.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:17 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:18.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:17 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:18.232 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:17 smithi112 ceph-mon[29186]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:18.232 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:17 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:18.232 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:17 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:18.315 DEBUG:teuthology.orchestra.run.smithi187:osd.8> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.8.service 2024-04-24T23:10:18.317 INFO:tasks.cephadm:Deploying osd.9 on smithi112 with /dev/vg_nvme/lv_1... 2024-04-24T23:10:18.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:10:18.570 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:19.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:18 smithi112 ceph-mon[29186]: from='osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:10:19.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:18 smithi112 ceph-mon[29186]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:10:19.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:18 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:19.009 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:18 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:18 smithi150 ceph-mon[32247]: from='osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:10:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:18 smithi150 ceph-mon[32247]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:10:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:18 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:18 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:19.111 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:18 smithi187 ceph-mon[33292]: from='osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:10:19.111 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:18 smithi187 ceph-mon[33292]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-24T23:10:19.111 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:18 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:19.111 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:18 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: from='osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:10:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: from='osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:10:20.095 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:20.096 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:10:19.818+0000 7f5ed282b700 -1 osd.8 0 waiting for initial osdmap 2024-04-24T23:10:20.096 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:10:19 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:10:19.827+0000 7f5ece61e700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:10:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-24T23:10:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: from='osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:10:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: osdmap e58: 9 total, 8 up, 9 in 2024-04-24T23:10:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:10:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:10:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: osdmap e59: 9 total, 8 up, 9 in 2024-04-24T23:10:20.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:10:20.606 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:10:20.607 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-24T23:10:20.607 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-24T23:10:20.607 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115212 s, 910 MB/s 2024-04-24T23:10:20.607 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-24T23:10:21.154 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:20 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:21.154 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:20 smithi187 ceph-mon[33292]: osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] boot 2024-04-24T23:10:21.155 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:20 smithi187 ceph-mon[33292]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:10:21.155 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:20 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:20 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:20 smithi112 ceph-mon[29186]: osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] boot 2024-04-24T23:10:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:20 smithi112 ceph-mon[29186]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:10:21.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:20 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:21.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:20 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:21.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:20 smithi150 ceph-mon[32247]: osd.8 [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] boot 2024-04-24T23:10:21.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:20 smithi150 ceph-mon[32247]: osdmap e60: 9 total, 9 up, 9 in 2024-04-24T23:10:21.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:20 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-24T23:10:21.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-04-24T23:10:21.657 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:10:22.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:21 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:10:22.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:21 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:10:22.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:21 smithi112 ceph-mon[29186]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:22.043 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:21 smithi112 ceph-mon[29186]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:10:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:21 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:10:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:21 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:10:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:21 smithi187 ceph-mon[33292]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:21 smithi187 ceph-mon[33292]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:10:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:21 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:10:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:21 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:10:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:21 smithi150 ceph-mon[32247]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-24T23:10:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:21 smithi150 ceph-mon[32247]: osdmap e61: 9 total, 9 up, 9 in 2024-04-24T23:10:23.125 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi187 2024-04-24T23:10:23.125 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.125 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.125 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi187 to 5696M 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.126 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:22 smithi112 ceph-mon[29186]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi187 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi187 to 5696M 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:23.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:23.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:23.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:23.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:22 smithi187 ceph-mon[33292]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi187 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi187 to 5696M 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:23.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:23.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:23.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:23.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:23.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:23.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:22 smithi150 ceph-mon[32247]: osdmap e62: 9 total, 9 up, 9 in 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 -- 172.21.15.112:0/3230331089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 msgr2=0x7f6a2410a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 --2- 172.21.15.112:0/3230331089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 0x7f6a2410a700 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00b0d0 tx=0x7f6a1c00b3e0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 -- 172.21.15.112:0/3230331089 shutdown_connections 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 --2- 172.21.15.112:0/3230331089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 0x7f6a2410a700 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 --2- 172.21.15.112:0/3230331089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a241022b0 0x7f6a241026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 --2- 172.21.15.112:0/3230331089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a241034b0 0x7f6a24103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 -- 172.21.15.112:0/3230331089 >> 172.21.15.112:0/3230331089 conn(0x7f6a240fd850 msgr2=0x7f6a240ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 -- 172.21.15.112:0/3230331089 shutdown_connections 2024-04-24T23:10:23.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 -- 172.21.15.112:0/3230331089 wait complete. 2024-04-24T23:10:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.354+0000 7f6a2af01700 1 Processor -- start 2024-04-24T23:10:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2af01700 1 -- start start 2024-04-24T23:10:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2af01700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a241022b0 0x7f6a24075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2af01700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a241034b0 0x7f6a24074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2af01700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 0x7f6a24074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2af01700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e030 con 0x7f6a24103e50 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2af01700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410deb0 con 0x7f6a241022b0 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2af01700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2410e1b0 con 0x7f6a241034b0 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 0x7f6a24074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 0x7f6a24074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48084/0 (socket says 172.21.15.112:48084) 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 -- 172.21.15.112:0/501735488 learned_addr learned my addr 172.21.15.112:0/501735488 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a28c9d700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a241022b0 0x7f6a24075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a23fff700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a241034b0 0x7f6a24074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 -- 172.21.15.112:0/501735488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a241034b0 msgr2=0x7f6a24074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a241034b0 0x7f6a24074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 -- 172.21.15.112:0/501735488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a241022b0 msgr2=0x7f6a24075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a241022b0 0x7f6a24075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 -- 172.21.15.112:0/501735488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a24075410 con 0x7f6a24103e50 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a23fff700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a241034b0 0x7f6a24074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:23.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2949e700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 0x7f6a24074b60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c007dc0 tx=0x7f6a1c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a21ffb700 1 -- 172.21.15.112:0/501735488 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c01f020 con 0x7f6a24103e50 2024-04-24T23:10:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a21ffb700 1 -- 172.21.15.112:0/501735488 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6a1c003e10 con 0x7f6a24103e50 2024-04-24T23:10:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a241a2630 con 0x7f6a24103e50 2024-04-24T23:10:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.355+0000 7f6a21ffb700 1 -- 172.21.15.112:0/501735488 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c01b880 con 0x7f6a24103e50 2024-04-24T23:10:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.356+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241a2a80 con 0x7f6a24103e50 2024-04-24T23:10:23.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.356+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a24068f20 con 0x7f6a24103e50 2024-04-24T23:10:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.358+0000 7f6a21ffb700 1 -- 172.21.15.112:0/501735488 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6a1c022020 con 0x7f6a24103e50 2024-04-24T23:10:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.358+0000 7f6a21ffb700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a140a2ca0 0x7f6a140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.358+0000 7f6a28c9d700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a140a2ca0 0x7f6a140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.358+0000 7f6a21ffb700 1 -- 172.21.15.112:0/501735488 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f6a1c0cfa50 con 0x7f6a24103e50 2024-04-24T23:10:23.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.359+0000 7f6a28c9d700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a140a2ca0 0x7f6a140a5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005950 tx=0x7f6a1800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:23.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.362+0000 7f6a21ffb700 1 -- 172.21.15.112:0/501735488 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6a1c029020 con 0x7f6a24103e50 2024-04-24T23:10:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:23.570+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6a24062670 con 0x7f6a140a2ca0 2024-04-24T23:10:24.194 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:23 smithi112 ceph-mon[29186]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:24.195 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:10:24.195 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:10:24.195 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:24.195 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:23 smithi112 ceph-mon[29186]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:10:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:23 smithi187 ceph-mon[33292]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:10:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:10:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:23 smithi187 ceph-mon[33292]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:10:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:23 smithi150 ceph-mon[32247]: pgmap v199: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:10:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:10:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:23 smithi150 ceph-mon[32247]: osdmap e63: 9 total, 9 up, 9 in 2024-04-24T23:10:25.176 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:24 smithi112 ceph-mon[29186]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:25.176 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:24 smithi112 ceph-mon[29186]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:10:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:24 smithi187 ceph-mon[33292]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:24 smithi187 ceph-mon[33292]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:10:25.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:24 smithi150 ceph-mon[32247]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:25.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:24 smithi150 ceph-mon[32247]: osdmap e64: 9 total, 9 up, 9 in 2024-04-24T23:10:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:25 smithi187 ceph-mon[33292]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:26.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:25 smithi112 ceph-mon[29186]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:25 smithi150 ceph-mon[32247]: pgmap v202: 1 pgs: 1 peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:27.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:26 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2341609662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6e8af2d-787c-472a-973b-da754768b4e6"}]: dispatch 2024-04-24T23:10:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:26 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2341609662' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6e8af2d-787c-472a-973b-da754768b4e6"}]': finished 2024-04-24T23:10:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:26 smithi112 ceph-mon[29186]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:10:27.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:26 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2341609662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6e8af2d-787c-472a-973b-da754768b4e6"}]: dispatch 2024-04-24T23:10:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:26 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2341609662' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6e8af2d-787c-472a-973b-da754768b4e6"}]': finished 2024-04-24T23:10:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:26 smithi187 ceph-mon[33292]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:10:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:26 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2341609662' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6e8af2d-787c-472a-973b-da754768b4e6"}]: dispatch 2024-04-24T23:10:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:26 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2341609662' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6e8af2d-787c-472a-973b-da754768b4e6"}]': finished 2024-04-24T23:10:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:26 smithi150 ceph-mon[32247]: osdmap e65: 10 total, 9 up, 10 in 2024-04-24T23:10:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:27 smithi187 ceph-mon[33292]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:10:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:27 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/925586438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:28.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:27 smithi112 ceph-mon[29186]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:10:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:27 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/925586438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:27 smithi150 ceph-mon[32247]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:10:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:27 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/925586438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:29 smithi187 ceph-mon[33292]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:10:30.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:29 smithi112 ceph-mon[29186]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:10:30.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:29 smithi150 ceph-mon[32247]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-24T23:10:32.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:31 smithi112 ceph-mon[29186]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:10:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:31 smithi187 ceph-mon[33292]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:10:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:31 smithi150 ceph-mon[32247]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-24T23:10:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:33 smithi187 ceph-mon[33292]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:10:34.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:33 smithi112 ceph-mon[29186]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:10:34.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:33 smithi150 ceph-mon[32247]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-24T23:10:35.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:34 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:10:35.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:34 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:35.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:34 smithi187 ceph-mon[33292]: Deploying daemon osd.9 on smithi112 2024-04-24T23:10:35.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:34 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:10:35.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:34 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:35.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:34 smithi112 ceph-mon[29186]: Deploying daemon osd.9 on smithi112 2024-04-24T23:10:35.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:34 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-24T23:10:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:34 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:34 smithi150 ceph-mon[32247]: Deploying daemon osd.9 on smithi112 2024-04-24T23:10:36.216 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:35 smithi112 ceph-mon[29186]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:10:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:35 smithi187 ceph-mon[33292]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:10:36.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:35 smithi150 ceph-mon[32247]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-24T23:10:38.228 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:37 smithi112 ceph-mon[29186]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:38.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:37 smithi187 ceph-mon[33292]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:37 smithi150 ceph-mon[32247]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:40.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:39 smithi187 ceph-mon[33292]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:40.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:39 smithi112 ceph-mon[29186]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:40.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:39 smithi150 ceph-mon[32247]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:41.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:41.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:41.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:41 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:41 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:41.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:41.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:41 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:42.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:42 smithi112 ceph-mon[29186]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:42 smithi150 ceph-mon[32247]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:42.485 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:42 smithi187 ceph-mon[33292]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:43.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.727+0000 7f6a21ffb700 1 -- 172.21.15.112:0/501735488 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6a24062670 con 0x7f6a140a2ca0 2024-04-24T23:10:43.730 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 9 on host 'smithi112' 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a140a2ca0 msgr2=0x7f6a140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a140a2ca0 0x7f6a140a5160 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005950 tx=0x7f6a1800a300 comp rx=0 tx=0).stop 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 msgr2=0x7f6a24074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 0x7f6a24074b60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c007dc0 tx=0x7f6a1c007df0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 shutdown_connections 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a140a2ca0 0x7f6a140a5160 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a241022b0 0x7f6a24075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a24103e50 0x7f6a24074b60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 --2- 172.21.15.112:0/501735488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a241034b0 0x7f6a24074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 >> 172.21.15.112:0/501735488 conn(0x7f6a240fd850 msgr2=0x7f6a24106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:43.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 shutdown_connections 2024-04-24T23:10:43.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:10:43.730+0000 7f6a2af01700 1 -- 172.21.15.112:0/501735488 wait complete. 2024-04-24T23:10:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:44 smithi112 ceph-mon[29186]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:44 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:44 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:44 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:44 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:44 smithi112 ceph-mon[29186]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:10:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:44 smithi150 ceph-mon[32247]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:44 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:44 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:44 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:44 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:44 smithi150 ceph-mon[32247]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:10:44.394 DEBUG:teuthology.orchestra.run.smithi112:osd.9> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.9.service 2024-04-24T23:10:44.397 INFO:tasks.cephadm:Deploying osd.10 on smithi150 with /dev/vg_nvme/lv_1... 2024-04-24T23:10:44.397 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:10:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:44 smithi187 ceph-mon[33292]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:44 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:44 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:44 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:44 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:44 smithi187 ceph-mon[33292]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-24T23:10:44.644 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:10:46.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:45 smithi150 ceph-mon[32247]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:46.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:45 smithi150 ceph-mon[32247]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:10:46.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:45 smithi150 ceph-mon[32247]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:10:46.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:45 smithi150 ceph-mon[32247]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:10:46.138 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:45 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:45 smithi187 ceph-mon[33292]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:45 smithi187 ceph-mon[33292]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:10:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:45 smithi187 ceph-mon[33292]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:10:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:45 smithi187 ceph-mon[33292]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:10:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:45 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:45 smithi112 ceph-mon[29186]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:46.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:45 smithi112 ceph-mon[29186]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-24T23:10:46.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:45 smithi112 ceph-mon[29186]: osdmap e66: 10 total, 9 up, 10 in 2024-04-24T23:10:46.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:45 smithi112 ceph-mon[29186]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-24T23:10:46.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:45 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:46.734 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:10:46.734 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:10:46.734 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-04-24T23:10:46.734 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-04-24T23:10:46.734 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013773 s, 761 MB/s 2024-04-24T23:10:46.734 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-04-24T23:10:46.782 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:10:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:10:46.527+0000 7f3070398700 -1 osd.9 0 waiting for initial osdmap 2024-04-24T23:10:46.783 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:10:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:10:46.539+0000 7f3068183700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:10:47.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:46 smithi150 ceph-mon[32247]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:10:47.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:46 smithi150 ceph-mon[32247]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:10:47.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:46 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:46 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:46 smithi150 ceph-mon[32247]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' 2024-04-24T23:10:47.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:46 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.143 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:46 smithi112 ceph-mon[29186]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:10:47.143 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:46 smithi112 ceph-mon[29186]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:10:47.143 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:46 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.143 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:46 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.143 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:46 smithi112 ceph-mon[29186]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' 2024-04-24T23:10:47.143 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:46 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:46 smithi187 ceph-mon[33292]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-24T23:10:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:46 smithi187 ceph-mon[33292]: osdmap e67: 10 total, 9 up, 10 in 2024-04-24T23:10:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:46 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:46 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:46 smithi187 ceph-mon[33292]: from='osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852]' entity='osd.9' 2024-04-24T23:10:47.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:46 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.458 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-04-24T23:10:47.719 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:10:47.998 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:10:47.998 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:10:47.998 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] boot 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:47.999 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:48.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:48.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:48.000 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:10:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:10:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] boot 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:10:48.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:48.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:48.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:48.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:48.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:10:48.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:10:48.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-24T23:10:48.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:48.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:48.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:48.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:10:48.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:48.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:48.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: osd.9 [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] boot 2024-04-24T23:10:48.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: osdmap e68: 10 total, 10 up, 10 in 2024-04-24T23:10:48.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-24T23:10:48.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:48.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:48.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:48.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:48.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:10:49.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:48 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi112 2024-04-24T23:10:49.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:48 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-24T23:10:49.254 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:48 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi112 2024-04-24T23:10:49.254 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:48 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-24T23:10:49.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:48 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi112 2024-04-24T23:10:49.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:48 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-24T23:10:49.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.481+0000 7f52bb12b700 1 -- 172.21.15.150:0/124928264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103e30 msgr2=0x7f52b410a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:49.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.481+0000 7f52bb12b700 1 --2- 172.21.15.150:0/124928264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103e30 0x7f52b410a6e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00b0d0 tx=0x7f52ac00b3e0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.483 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.481+0000 7f52bb12b700 1 -- 172.21.15.150:0/124928264 shutdown_connections 2024-04-24T23:10:49.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.481+0000 7f52bb12b700 1 --2- 172.21.15.150:0/124928264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103e30 0x7f52b410a6e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.481+0000 7f52bb12b700 1 --2- 172.21.15.150:0/124928264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b4102290 0x7f52b41026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.481+0000 7f52bb12b700 1 --2- 172.21.15.150:0/124928264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b4103490 0x7f52b41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.481+0000 7f52bb12b700 1 -- 172.21.15.150:0/124928264 >> 172.21.15.150:0/124928264 conn(0x7f52b40fd830 msgr2=0x7f52b40ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:49.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.481+0000 7f52bb12b700 1 -- 172.21.15.150:0/124928264 shutdown_connections 2024-04-24T23:10:49.484 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 -- 172.21.15.150:0/124928264 wait complete. 2024-04-24T23:10:49.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 Processor -- start 2024-04-24T23:10:49.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 -- start start 2024-04-24T23:10:49.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b4102290 0x7f52b4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:49.486 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103490 0x7f52b4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b4103e30 0x7f52b419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52b410e010 con 0x7f52b4102290 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52b410de90 con 0x7f52b4103490 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52bb12b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52b410e190 con 0x7f52b4103e30 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103490 0x7f52b4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103490 0x7f52b4198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:58478/0 (socket says 172.21.15.150:58478) 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52b3fff700 1 -- 172.21.15.150:0/343606122 learned_addr learned my addr 172.21.15.150:0/343606122 (peer_addr_for_me v2:172.21.15.150:0/0) 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.482+0000 7f52b8ec7700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b4102290 0x7f52b4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b96c8700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b4103e30 0x7f52b419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b3fff700 1 -- 172.21.15.150:0/343606122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b4103e30 msgr2=0x7f52b419fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b3fff700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b4103e30 0x7f52b419fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.487 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b3fff700 1 -- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b4102290 msgr2=0x7f52b4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b3fff700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b4102290 0x7f52b4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b3fff700 1 -- 172.21.15.150:0/343606122 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b41a0160 con 0x7f52b4103490 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b96c8700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b4103e30 0x7f52b419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b3fff700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103490 0x7f52b4198a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52a000e9a0 tx=0x7f52a000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b1ffb700 1 -- 172.21.15.150:0/343606122 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a000cbd0 con 0x7f52b4103490 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b1ffb700 1 -- 172.21.15.150:0/343606122 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f52a000cd70 con 0x7f52b4103490 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52b1ffb700 1 -- 172.21.15.150:0/343606122 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0018840 con 0x7f52b4103490 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b41a0450 con 0x7f52b4103490 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.483+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b41a09e0 con 0x7f52b4103490 2024-04-24T23:10:49.488 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.484+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b4068f20 con 0x7f52b4103490 2024-04-24T23:10:49.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.485+0000 7f52b1ffb700 1 -- 172.21.15.150:0/343606122 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f52a0010a50 con 0x7f52b4103490 2024-04-24T23:10:49.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.486+0000 7f52b1ffb700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52a40a2ca0 0x7f52a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:49.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.486+0000 7f52b1ffb700 1 -- 172.21.15.150:0/343606122 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f52a00cee80 con 0x7f52b4103490 2024-04-24T23:10:49.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.486+0000 7f52b8ec7700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52a40a2ca0 0x7f52a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:49.489 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.487+0000 7f52b8ec7700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52a40a2ca0 0x7f52a40a5160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52b41032f0 tx=0x7f52a800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:49.491 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.490+0000 7f52b1ffb700 1 -- 172.21.15.150:0/343606122 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f52a009c5e0 con 0x7f52b4103490 2024-04-24T23:10:49.695 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:10:49.693+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f52b4062730 con 0x7f52a40a2ca0 2024-04-24T23:10:50.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:49 smithi150 ceph-mon[32247]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:50.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:49 smithi150 ceph-mon[32247]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:10:50.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:10:50.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:10:50.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:49 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:49 smithi187 ceph-mon[33292]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:49 smithi187 ceph-mon[33292]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:10:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:10:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:10:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:49 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:50.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:49 smithi112 ceph-mon[29186]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:49 smithi112 ceph-mon[29186]: osdmap e69: 10 total, 10 up, 10 in 2024-04-24T23:10:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:10:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:10:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:49 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:50 smithi187 ceph-mon[33292]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:51.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:50 smithi112 ceph-mon[29186]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:51.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:50 smithi150 ceph-mon[32247]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:51 smithi187 ceph-mon[33292]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:51 smithi112 ceph-mon[29186]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:51 smithi150 ceph-mon[32247]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:52 smithi187 ceph-mon[33292]: from='client.? 172.21.15.150:0/589856199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]: dispatch 2024-04-24T23:10:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:52 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]: dispatch 2024-04-24T23:10:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:52 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]': finished 2024-04-24T23:10:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:52 smithi187 ceph-mon[33292]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:10:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:52 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:10:53.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:52 smithi112 ceph-mon[29186]: from='client.? 172.21.15.150:0/589856199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]: dispatch 2024-04-24T23:10:53.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:52 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]: dispatch 2024-04-24T23:10:53.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:52 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]': finished 2024-04-24T23:10:53.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:52 smithi112 ceph-mon[29186]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:10:53.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:52 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:10:53.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:52 smithi150 ceph-mon[32247]: from='client.? 172.21.15.150:0/589856199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]: dispatch 2024-04-24T23:10:53.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:52 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]: dispatch 2024-04-24T23:10:53.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:52 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0114d00e-ffc8-4e0d-b79d-c87a604b5cc3"}]': finished 2024-04-24T23:10:53.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:52 smithi150 ceph-mon[32247]: osdmap e70: 11 total, 10 up, 11 in 2024-04-24T23:10:53.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:52 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:10:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:53 smithi187 ceph-mon[33292]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:53 smithi187 ceph-mon[33292]: from='client.? 172.21.15.150:0/3143766979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:54.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:53 smithi112 ceph-mon[29186]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.150:0/3143766979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:54.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:53 smithi150 ceph-mon[32247]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:53 smithi150 ceph-mon[32247]: from='client.? 172.21.15.150:0/3143766979' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:10:56.208 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:55 smithi150 ceph-mon[32247]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:55 smithi187 ceph-mon[33292]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:56.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:55 smithi112 ceph-mon[29186]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:57.891 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:57 smithi150 ceph-mon[32247]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:57 smithi187 ceph-mon[33292]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:10:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:57 smithi112 ceph-mon[29186]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:00.182 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:10:59 smithi150 ceph-mon[32247]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:10:59 smithi187 ceph-mon[33292]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:00.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:10:59 smithi112 ceph-mon[29186]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:01.160 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:00 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:11:01.161 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:00 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:01.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:00 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:11:01.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:00 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:01.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:00 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-24T23:11:01.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:00 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:02.224 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:01 smithi150 ceph-mon[32247]: Deploying daemon osd.10 on smithi150 2024-04-24T23:11:02.225 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:01 smithi150 ceph-mon[32247]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:02.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:01 smithi187 ceph-mon[33292]: Deploying daemon osd.10 on smithi150 2024-04-24T23:11:02.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:01 smithi187 ceph-mon[33292]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:02.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:01 smithi112 ceph-mon[29186]: Deploying daemon osd.10 on smithi150 2024-04-24T23:11:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:01 smithi112 ceph-mon[29186]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:04.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:03 smithi150 ceph-mon[32247]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:04.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:03 smithi187 ceph-mon[33292]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:04.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:03 smithi112 ceph-mon[29186]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:05 smithi187 ceph-mon[33292]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:06.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:05 smithi112 ceph-mon[29186]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:06.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:05 smithi150 ceph-mon[32247]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:07.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:06 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:07.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:06 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:07.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:06 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:06 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:07.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:06 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:07.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:06 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:06 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:06 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:06 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:08.261 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:07 smithi150 ceph-mon[32247]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:08.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:07 smithi112 ceph-mon[29186]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:08.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:07 smithi187 ceph-mon[33292]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:09.715 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 10 on host 'smithi150' 2024-04-24T23:11:09.715 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.712+0000 7f52b1ffb700 1 -- 172.21.15.150:0/343606122 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f52b4062730 con 0x7f52a40a2ca0 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52a40a2ca0 msgr2=0x7f52a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52a40a2ca0 0x7f52a40a5160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52b41032f0 tx=0x7f52a800b410 comp rx=0 tx=0).stop 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103490 msgr2=0x7f52b4198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103490 0x7f52b4198a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52a000e9a0 tx=0x7f52a000ecb0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 shutdown_connections 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52a40a2ca0 0x7f52a40a5160 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b4103e30 0x7f52b419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52b4102290 0x7f52b4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 --2- 172.21.15.150:0/343606122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52b4103490 0x7f52b4198a40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:09.717 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 >> 172.21.15.150:0/343606122 conn(0x7f52b40fd830 msgr2=0x7f52b4106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:09.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 shutdown_connections 2024-04-24T23:11:09.718 INFO:teuthology.orchestra.run.smithi150.stderr:2024-04-24T23:11:09.715+0000 7f52bb12b700 1 -- 172.21.15.150:0/343606122 wait complete. 2024-04-24T23:11:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:09 smithi112 ceph-mon[29186]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:09 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:09 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:09 smithi112 ceph-mon[29186]: from='osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:11:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:09 smithi112 ceph-mon[29186]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:11:10.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:09 smithi150 ceph-mon[32247]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:09 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:09 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:09 smithi150 ceph-mon[32247]: from='osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:11:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:09 smithi150 ceph-mon[32247]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:11:10.333 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:09 smithi187 ceph-mon[33292]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:10.333 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:09 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:10.333 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:09 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:10.333 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:09 smithi187 ceph-mon[33292]: from='osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:11:10.333 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:09 smithi187 ceph-mon[33292]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-24T23:11:10.447 DEBUG:teuthology.orchestra.run.smithi150:osd.10> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.10.service 2024-04-24T23:11:10.449 INFO:tasks.cephadm:Deploying osd.11 on smithi187 with /dev/vg_nvme/lv_1... 2024-04-24T23:11:10.450 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-24T23:11:10.688 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:11:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:11 smithi187 ceph-mon[33292]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:11:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:11 smithi187 ceph-mon[33292]: from='osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:11:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:11 smithi187 ceph-mon[33292]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:11:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:11 smithi187 ceph-mon[33292]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:11:11.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:11 smithi187 ceph-mon[33292]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:11.753 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:11.753 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:11.754 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:11 smithi150 ceph-mon[32247]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:11:11.754 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:11 smithi150 ceph-mon[32247]: from='osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:11:11.754 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:11 smithi150 ceph-mon[32247]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:11:11.754 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:11.754 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:11 smithi150 ceph-mon[32247]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:11:11.754 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:11 smithi150 ceph-mon[32247]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:11.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:11.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:11.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:11 smithi112 ceph-mon[29186]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-24T23:11:11.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:11 smithi112 ceph-mon[29186]: from='osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:11:11.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:11 smithi112 ceph-mon[29186]: osdmap e71: 11 total, 10 up, 11 in 2024-04-24T23:11:11.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:11.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:11 smithi112 ceph-mon[29186]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-04-24T23:11:11.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:11 smithi112 ceph-mon[29186]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:12.523 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:11:12 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:11:12.382+0000 7fc3665d7700 -1 osd.10 0 waiting for initial osdmap 2024-04-24T23:11:12.523 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:11:12 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:11:12.393+0000 7fc35e3c2700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:11:12.687 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-24T23:11:12.687 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-24T23:11:12.687 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2024-04-24T23:11:12.687 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2024-04-24T23:11:12.687 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114195 s, 918 MB/s 2024-04-24T23:11:12.687 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2024-04-24T23:11:12.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:12 smithi150 ceph-mon[32247]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:11:12.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:12 smithi150 ceph-mon[32247]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:11:12.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:12 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:12.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:12 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:12.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:12 smithi150 ceph-mon[32247]: from='osd.10 ' entity='osd.10' 2024-04-24T23:11:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:12 smithi187 ceph-mon[33292]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:11:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:12 smithi187 ceph-mon[33292]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:11:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:12 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:12 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:12.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:12 smithi187 ceph-mon[33292]: from='osd.10 ' entity='osd.10' 2024-04-24T23:11:13.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:12 smithi112 ceph-mon[29186]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-04-24T23:11:13.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:12 smithi112 ceph-mon[29186]: osdmap e72: 11 total, 10 up, 11 in 2024-04-24T23:11:13.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:12 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:13.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:12 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:13.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:12 smithi112 ceph-mon[29186]: from='osd.10 ' entity='osd.10' 2024-04-24T23:11:13.360 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi187:vg_nvme/lv_1 2024-04-24T23:11:13.595 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:11:13.953 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:13 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:11:13.953 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:13 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:11:13.953 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:13.953 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:13 smithi187 ceph-mon[33292]: osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] boot 2024-04-24T23:11:13.953 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:13 smithi187 ceph-mon[33292]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:11:13.953 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:13.953 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:13 smithi187 ceph-mon[33292]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:13.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:13 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:11:13.994 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:13 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:11:13.995 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:13.995 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:13 smithi150 ceph-mon[32247]: osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] boot 2024-04-24T23:11:13.995 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:13 smithi150 ceph-mon[32247]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:11:13.995 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:13.995 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:13 smithi150 ceph-mon[32247]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:13 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:11:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:13 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:11:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:13 smithi112 ceph-mon[29186]: osd.10 [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] boot 2024-04-24T23:11:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:13 smithi112 ceph-mon[29186]: osdmap e73: 11 total, 11 up, 11 in 2024-04-24T23:11:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-24T23:11:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:13 smithi112 ceph-mon[29186]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-24T23:11:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:11:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:14.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:14.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:14 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:15.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:15.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:15.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:14 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: osdmap e74: 11 total, 11 up, 11 in 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:15.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:15.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:15.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:15.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:15.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:14 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:15.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 -- 172.21.15.187:0/767150269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 msgr2=0x7f4134106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 --2- 172.21.15.187:0/767150269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 0x7f4134106550 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f412c00b0a0 tx=0x7f412c00b3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 -- 172.21.15.187:0/767150269 shutdown_connections 2024-04-24T23:11:15.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 --2- 172.21.15.187:0/767150269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4134100580 0x7f41341009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 --2- 172.21.15.187:0/767150269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 0x7f4134106550 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 --2- 172.21.15.187:0/767150269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41341012f0 0x7f4134101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 -- 172.21.15.187:0/767150269 >> 172.21.15.187:0/767150269 conn(0x7f41340fbee0 msgr2=0x7f41340fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 -- 172.21.15.187:0/767150269 shutdown_connections 2024-04-24T23:11:15.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.248+0000 7f413b3a5700 1 -- 172.21.15.187:0/767150269 wait complete. 2024-04-24T23:11:15.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f413b3a5700 1 Processor -- start 2024-04-24T23:11:15.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f413b3a5700 1 -- start start 2024-04-24T23:11:15.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f413b3a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4134100580 0x7f413419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f413b3a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41341012f0 0x7f413419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f413b3a5700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 0x7f41341a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f413b3a5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4134114620 con 0x7f41341012f0 2024-04-24T23:11:15.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f413b3a5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f41341144a0 con 0x7f4134100580 2024-04-24T23:11:15.252 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f413b3a5700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41341147a0 con 0x7f4134101cb0 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 0x7f41341a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 0x7f41341a5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:50722/0 (socket says 172.21.15.187:50722) 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 -- 172.21.15.187:0/3369742008 learned_addr learned my addr 172.21.15.187:0/3369742008 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139141700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4134100580 0x7f413419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4138940700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41341012f0 0x7f413419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 -- 172.21.15.187:0/3369742008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4134100580 msgr2=0x7f413419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4134100580 0x7f413419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 -- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41341012f0 msgr2=0x7f413419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41341012f0 0x7f413419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 -- 172.21.15.187:0/3369742008 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41341a65e0 con 0x7f4134101cb0 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.249+0000 7f4139942700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 0x7f41341a5f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f412c007d10 tx=0x7f412c007d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.250+0000 7f412a7fc700 1 -- 172.21.15.187:0/3369742008 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c003e10 con 0x7f4134101cb0 2024-04-24T23:11:15.253 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.250+0000 7f412a7fc700 1 -- 172.21.15.187:0/3369742008 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f412c022070 con 0x7f4134101cb0 2024-04-24T23:11:15.254 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.250+0000 7f412a7fc700 1 -- 172.21.15.187:0/3369742008 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c01b970 con 0x7f4134101cb0 2024-04-24T23:11:15.254 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.250+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41341a6870 con 0x7f4134101cb0 2024-04-24T23:11:15.254 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.250+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a6c70 con 0x7f4134101cb0 2024-04-24T23:11:15.254 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.251+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4134068f20 con 0x7f4134101cb0 2024-04-24T23:11:15.254 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.252+0000 7f412a7fc700 1 -- 172.21.15.187:0/3369742008 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f412c01db90 con 0x7f4134101cb0 2024-04-24T23:11:15.255 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.252+0000 7f412a7fc700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f41200a2ca0 0x7f41200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.255 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.252+0000 7f412a7fc700 1 -- 172.21.15.187:0/3369742008 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f412c00f020 con 0x7f4134101cb0 2024-04-24T23:11:15.255 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.253+0000 7f4139141700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f41200a2ca0 0x7f41200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.255 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.253+0000 7f4139141700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f41200a2ca0 0x7f41200a5160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f413000ac50 tx=0x7f413000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.256 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.256+0000 7f412a7fc700 1 -- 172.21.15.187:0/3369742008 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f412c09d2f0 con 0x7f4134101cb0 2024-04-24T23:11:15.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:15.460+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4134062670 con 0x7f41200a2ca0 2024-04-24T23:11:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:15 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi150 2024-04-24T23:11:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:15 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-24T23:11:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:15 smithi187 ceph-mon[33292]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:11:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:15 smithi187 ceph-mon[33292]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:11:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:11:15.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:16.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:15 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi150 2024-04-24T23:11:16.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:15 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-24T23:11:16.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:15 smithi112 ceph-mon[29186]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:11:16.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:15 smithi112 ceph-mon[29186]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:16.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:11:16.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:11:16.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:16.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:15 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi150 2024-04-24T23:11:16.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:15 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi150 to 4272M 2024-04-24T23:11:16.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:15 smithi150 ceph-mon[32247]: osdmap e75: 11 total, 11 up, 11 in 2024-04-24T23:11:16.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:15 smithi150 ceph-mon[32247]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:16.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:11:16.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:11:16.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:16.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:16 smithi187 ceph-mon[33292]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:16.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:16 smithi187 ceph-mon[33292]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:11:17.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:16 smithi112 ceph-mon[29186]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:17.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:16 smithi112 ceph-mon[29186]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:11:17.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:16 smithi150 ceph-mon[32247]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:17.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:16 smithi150 ceph-mon[32247]: osdmap e76: 11 total, 11 up, 11 in 2024-04-24T23:11:17.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:17 smithi187 ceph-mon[33292]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:11:17.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:17 smithi187 ceph-mon[33292]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:11:18.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:17 smithi112 ceph-mon[29186]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:11:18.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:17 smithi112 ceph-mon[29186]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:11:18.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:17 smithi150 ceph-mon[32247]: osdmap e77: 11 total, 11 up, 11 in 2024-04-24T23:11:18.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:17 smithi150 ceph-mon[32247]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:11:18.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:18 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/1817909919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]: dispatch 2024-04-24T23:11:18.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:18 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]: dispatch 2024-04-24T23:11:18.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:18 smithi187 ceph-mon[33292]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]': finished 2024-04-24T23:11:18.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:18 smithi187 ceph-mon[33292]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:11:18.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:18 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:19.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:18 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/1817909919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]: dispatch 2024-04-24T23:11:19.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:18 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]: dispatch 2024-04-24T23:11:19.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:18 smithi112 ceph-mon[29186]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]': finished 2024-04-24T23:11:19.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:18 smithi112 ceph-mon[29186]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:11:19.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:18 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:18 smithi150 ceph-mon[32247]: from='client.? 172.21.15.187:0/1817909919' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]: dispatch 2024-04-24T23:11:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:18 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]: dispatch 2024-04-24T23:11:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:18 smithi150 ceph-mon[32247]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "971bddea-134b-4d7e-9ae6-770a9129fdb1"}]': finished 2024-04-24T23:11:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:18 smithi150 ceph-mon[32247]: osdmap e78: 12 total, 11 up, 12 in 2024-04-24T23:11:19.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:18 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:19 smithi112 ceph-mon[29186]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-24T23:11:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:19 smithi112 ceph-mon[29186]: from='client.? 172.21.15.187:0/324912130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:11:20.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:19 smithi150 ceph-mon[32247]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-24T23:11:20.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:19 smithi150 ceph-mon[32247]: from='client.? 172.21.15.187:0/324912130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:11:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:19 smithi187 ceph-mon[33292]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-24T23:11:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:19 smithi187 ceph-mon[33292]: from='client.? 172.21.15.187:0/324912130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:11:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:21 smithi187 ceph-mon[33292]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:11:22.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:21 smithi112 ceph-mon[29186]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:11:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:21 smithi150 ceph-mon[32247]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:11:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:23 smithi187 ceph-mon[33292]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:24.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:23 smithi112 ceph-mon[29186]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:23 smithi150 ceph-mon[32247]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:26.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:25 smithi187 ceph-mon[33292]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:11:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:25 smithi112 ceph-mon[29186]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:11:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:26.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:25 smithi150 ceph-mon[32247]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-24T23:11:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:26.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:26 smithi187 ceph-mon[33292]: Deploying daemon osd.11 on smithi187 2024-04-24T23:11:27.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:26 smithi112 ceph-mon[29186]: Deploying daemon osd.11 on smithi187 2024-04-24T23:11:27.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:26 smithi150 ceph-mon[32247]: Deploying daemon osd.11 on smithi187 2024-04-24T23:11:28.010 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:27 smithi187 ceph-mon[33292]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:28.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:27 smithi112 ceph-mon[29186]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:27 smithi150 ceph-mon[32247]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:29.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:29 smithi187 ceph-mon[33292]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:30.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:29 smithi112 ceph-mon[29186]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:30.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:29 smithi150 ceph-mon[32247]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:32.098 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:31 smithi187 ceph-mon[33292]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:32.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:32.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:32.099 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:31 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:31 smithi112 ceph-mon[29186]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:31 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:31 smithi150 ceph-mon[32247]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:31 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:34.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:33 smithi187 ceph-mon[33292]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:34.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:33 smithi112 ceph-mon[29186]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:34.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:33 smithi150 ceph-mon[32247]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:35.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.529+0000 7f412a7fc700 1 -- 172.21.15.187:0/3369742008 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4134062670 con 0x7f41200a2ca0 2024-04-24T23:11:35.532 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 11 on host 'smithi187' 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f41200a2ca0 msgr2=0x7f41200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f41200a2ca0 0x7f41200a5160 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f413000ac50 tx=0x7f413000a380 comp rx=0 tx=0).stop 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 msgr2=0x7f41341a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 0x7f41341a5f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f412c007d10 tx=0x7f412c007d40 comp rx=0 tx=0).stop 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 shutdown_connections 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f41200a2ca0 0x7f41200a5160 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4134100580 0x7f413419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101cb0 0x7f41341a5f40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 --2- 172.21.15.187:0/3369742008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41341012f0 0x7f413419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:35.534 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 >> 172.21.15.187:0/3369742008 conn(0x7f41340fbee0 msgr2=0x7f4134102530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:35.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 shutdown_connections 2024-04-24T23:11:35.535 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:35.532+0000 7f413b3a5700 1 -- 172.21.15.187:0/3369742008 wait complete. 2024-04-24T23:11:35.902 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:35.902 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:35.902 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:35.903 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:35.903 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:35.903 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:35.903 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:11:35.903 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:11:35.903 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:35.903 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:35.903 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:35 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:36.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:36.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:36.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:36.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:11:36.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:11:36.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:35 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:36.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:35 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:36.263 DEBUG:teuthology.orchestra.run.smithi187:osd.11> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.11.service 2024-04-24T23:11:36.265 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-24T23:11:36.265 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:11:36.514 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:36.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:36 smithi187 ceph-mon[33292]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:11:36.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:36 smithi187 ceph-mon[33292]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:11:36.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:36 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:36.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:36 smithi187 ceph-mon[33292]: from='osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:11:36.799 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:36 smithi187 ceph-mon[33292]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:11:36.800 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:11:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:11:36.632+0000 7fc005837700 -1 osd.11 0 waiting for initial osdmap 2024-04-24T23:11:36.800 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:11:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:11:36.644+0000 7fc00162a700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:11:36.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:36 smithi112 ceph-mon[29186]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:11:36.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:36 smithi112 ceph-mon[29186]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:11:36.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:36 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:36.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:36 smithi112 ceph-mon[29186]: from='osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:11:36.955 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:36 smithi112 ceph-mon[29186]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:11:37.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:36 smithi150 ceph-mon[32247]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-24T23:11:37.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:36 smithi150 ceph-mon[32247]: osdmap e79: 12 total, 11 up, 12 in 2024-04-24T23:11:37.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:36 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:37.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:36 smithi150 ceph-mon[32247]: from='osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:11:37.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:36 smithi150 ceph-mon[32247]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-24T23:11:37.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:37 smithi187 ceph-mon[33292]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:11:37.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:37 smithi187 ceph-mon[33292]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:11:37.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:37.796 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:37 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:37.797 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:37 smithi187 ceph-mon[33292]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:37.887 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:37 smithi112 ceph-mon[29186]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:11:37.887 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:37 smithi112 ceph-mon[29186]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:11:37.888 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:37.888 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:37 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:37.888 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:37 smithi112 ceph-mon[29186]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:38.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:37 smithi150 ceph-mon[32247]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-24T23:11:38.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:37 smithi150 ceph-mon[32247]: osdmap e80: 12 total, 11 up, 12 in 2024-04-24T23:11:38.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:38.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:37 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:38.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:37 smithi150 ceph-mon[32247]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 -- 172.21.15.112:0/185915995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c1024a0 msgr2=0x7ff61c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 --2- 172.21.15.112:0/185915995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c1024a0 0x7ff61c108d50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c240 tx=0x7ff61400c550 comp rx=0 tx=0).stop 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 -- 172.21.15.112:0/185915995 shutdown_connections 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 --2- 172.21.15.112:0/185915995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c1024a0 0x7ff61c108d50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 --2- 172.21.15.112:0/185915995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c100900 0x7ff61c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 --2- 172.21.15.112:0/185915995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff61c101b00 0x7ff61c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 -- 172.21.15.112:0/185915995 >> 172.21.15.112:0/185915995 conn(0x7ff61c0fbec0 msgr2=0x7ff61c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 -- 172.21.15.112:0/185915995 shutdown_connections 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.256+0000 7ff62106b700 1 -- 172.21.15.112:0/185915995 wait complete. 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.257+0000 7ff62106b700 1 Processor -- start 2024-04-24T23:11:38.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.257+0000 7ff62106b700 1 -- start start 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff62106b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c100900 0x7ff61c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff62106b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c101b00 0x7ff61c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff62106b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff61c1024a0 0x7ff61c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff62106b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c10be10 con 0x7ff61c100900 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff62106b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c10bc90 con 0x7ff61c1024a0 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff62106b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c10bf90 con 0x7ff61c101b00 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c100900 0x7ff61c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c100900 0x7ff61c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35966/0 (socket says 172.21.15.112:35966) 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61a59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c101b00 0x7ff61c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 -- 172.21.15.112:0/822590358 learned_addr learned my addr 172.21.15.112:0/822590358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61b59e700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff61c1024a0 0x7ff61c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 -- 172.21.15.112:0/822590358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c101b00 msgr2=0x7ff61c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c101b00 0x7ff61c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 -- 172.21.15.112:0/822590358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff61c1024a0 msgr2=0x7ff61c19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff61c1024a0 0x7ff61c19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 -- 172.21.15.112:0/822590358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c1a01d0 con 0x7ff61c100900 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61b59e700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff61c1024a0 0x7ff61c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff61ad9d700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c100900 0x7ff61c198570 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b530 tx=0x7ff60c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.258+0000 7ff603fff700 1 -- 172.21.15.112:0/822590358 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c015020 con 0x7ff61c100900 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.259+0000 7ff603fff700 1 -- 172.21.15.112:0/822590358 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff60c011d30 con 0x7ff61c100900 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.259+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c1a04c0 con 0x7ff61c100900 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.259+0000 7ff603fff700 1 -- 172.21.15.112:0/822590358 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c00f2e0 con 0x7ff61c100900 2024-04-24T23:11:38.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.259+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c1a0a00 con 0x7ff61c100900 2024-04-24T23:11:38.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.260+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c068f20 con 0x7ff61c100900 2024-04-24T23:11:38.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.262+0000 7ff603fff700 1 -- 172.21.15.112:0/822590358 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff60c018020 con 0x7ff61c100900 2024-04-24T23:11:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.263+0000 7ff603fff700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6040a2c20 0x7ff6040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.263+0000 7ff61a59c700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6040a2c20 0x7ff6040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.263+0000 7ff603fff700 1 -- 172.21.15.112:0/822590358 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff60c0ce8e0 con 0x7ff61c100900 2024-04-24T23:11:38.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.263+0000 7ff61a59c700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6040a2c20 0x7ff6040a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff610005950 tx=0x7ff61000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.266+0000 7ff603fff700 1 -- 172.21.15.112:0/822590358 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff60c01f020 con 0x7ff61c100900 2024-04-24T23:11:38.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.474+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff61c1a0cc0 con 0x7ff61c100900 2024-04-24T23:11:38.477 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.475+0000 7ff603fff700 1 -- 172.21.15.112:0/822590358 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7ff60c09bce0 con 0x7ff61c100900 2024-04-24T23:11:38.477 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:11:38.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6040a2c20 msgr2=0x7ff6040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6040a2c20 0x7ff6040a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff610005950 tx=0x7ff61000b410 comp rx=0 tx=0).stop 2024-04-24T23:11:38.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c100900 msgr2=0x7ff61c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c100900 0x7ff61c198570 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00b530 tx=0x7ff60c00b840 comp rx=0 tx=0).stop 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 shutdown_connections 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6040a2c20 0x7ff6040a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff61c1024a0 0x7ff61c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff61c100900 0x7ff61c198570 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 --2- 172.21.15.112:0/822590358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c101b00 0x7ff61c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 >> 172.21.15.112:0/822590358 conn(0x7ff61c0fbec0 msgr2=0x7ff61c106f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 shutdown_connections 2024-04-24T23:11:38.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:38.477+0000 7ff62106b700 1 -- 172.21.15.112:0/822590358 wait complete. 2024-04-24T23:11:38.778 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:38 smithi112 ceph-mon[29186]: purged_snaps scrub starts 2024-04-24T23:11:38.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:38 smithi112 ceph-mon[29186]: purged_snaps scrub ok 2024-04-24T23:11:38.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:38 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:38.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:38 smithi112 ceph-mon[29186]: osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] boot 2024-04-24T23:11:38.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:38 smithi112 ceph-mon[29186]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:11:38.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:38 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:38.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:38 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/822590358' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:11:38.920 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:38 smithi187 ceph-mon[33292]: purged_snaps scrub starts 2024-04-24T23:11:38.920 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:38 smithi187 ceph-mon[33292]: purged_snaps scrub ok 2024-04-24T23:11:38.920 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:38 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:38.920 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:38 smithi187 ceph-mon[33292]: osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] boot 2024-04-24T23:11:38.920 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:38 smithi187 ceph-mon[33292]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:11:38.920 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:38 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:38.920 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:38 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/822590358' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:11:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:38 smithi150 ceph-mon[32247]: purged_snaps scrub starts 2024-04-24T23:11:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:38 smithi150 ceph-mon[32247]: purged_snaps scrub ok 2024-04-24T23:11:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:38 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:38 smithi150 ceph-mon[32247]: osd.11 [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] boot 2024-04-24T23:11:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:38 smithi150 ceph-mon[32247]: osdmap e81: 12 total, 12 up, 12 in 2024-04-24T23:11:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:38 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-24T23:11:39.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:38 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/822590358' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:11:39.128 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714000297,"num_in_osds":12,"osd_in_since":1714000278,"num_remapped_pgs":0} 2024-04-24T23:11:39.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:11:39.376 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:39.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:11:39.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:39.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:39.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:39.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:39.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:39.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:39.715 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:39.716 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:39.716 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:39.716 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:39.716 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:39 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:39.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:11:39.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:39.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:39.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:39.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:39.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:39.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:39.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:39.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:39.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:39.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:39.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:39 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: osdmap e82: 12 total, 12 up, 12 in 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:40.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:40.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:40.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:39 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:40.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:40 smithi112 ceph-mon[29186]: Detected new or changed devices on smithi187 2024-04-24T23:11:40.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:40 smithi112 ceph-mon[29186]: Adjusting osd_memory_target on smithi187 to 4272M 2024-04-24T23:11:40.969 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:40 smithi150 ceph-mon[32247]: Detected new or changed devices on smithi187 2024-04-24T23:11:40.969 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:40 smithi150 ceph-mon[32247]: Adjusting osd_memory_target on smithi187 to 4272M 2024-04-24T23:11:40.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:40 smithi187 ceph-mon[33292]: Detected new or changed devices on smithi187 2024-04-24T23:11:40.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:40 smithi187 ceph-mon[33292]: Adjusting osd_memory_target on smithi187 to 4272M 2024-04-24T23:11:41.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 -- 172.21.15.112:0/1365589086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc7406a8e0 msgr2=0x7efc7410f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 --2- 172.21.15.112:0/1365589086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc7406a8e0 0x7efc7410f0f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00b0a0 tx=0x7efc6c00b3b0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 -- 172.21.15.112:0/1365589086 shutdown_connections 2024-04-24T23:11:41.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 --2- 172.21.15.112:0/1365589086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc7406a8e0 0x7efc7410f0f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 --2- 172.21.15.112:0/1365589086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc741082b0 0x7efc74108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 --2- 172.21.15.112:0/1365589086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc74069f40 0x7efc7406a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 -- 172.21.15.112:0/1365589086 >> 172.21.15.112:0/1365589086 conn(0x7efc740780e0 msgr2=0x7efc740784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 -- 172.21.15.112:0/1365589086 shutdown_connections 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 -- 172.21.15.112:0/1365589086 wait complete. 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.055+0000 7efc79af2700 1 Processor -- start 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc79af2700 1 -- start start 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc79af2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc74069f40 0x7efc7419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc79af2700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7406a8e0 0x7efc7419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc79af2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc741082b0 0x7efc74198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc79af2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc74112640 con 0x7efc74069f40 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc79af2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efc741124c0 con 0x7efc741082b0 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc79af2700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc741127c0 con 0x7efc7406a8e0 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc74069f40 0x7efc7419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:41.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc74069f40 0x7efc7419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47386/0 (socket says 172.21.15.112:47386) 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 -- 172.21.15.112:0/1711373984 learned_addr learned my addr 172.21.15.112:0/1711373984 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc73fff700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc741082b0 0x7efc74198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc72ffd700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7406a8e0 0x7efc7419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 -- 172.21.15.112:0/1711373984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7406a8e0 msgr2=0x7efc7419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7406a8e0 0x7efc7419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 -- 172.21.15.112:0/1711373984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc741082b0 msgr2=0x7efc74198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc741082b0 0x7efc74198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 -- 172.21.15.112:0/1711373984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc74198fe0 con 0x7efc74069f40 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc73fff700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc741082b0 0x7efc74198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:11:41.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.056+0000 7efc737fe700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc74069f40 0x7efc7419e680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c9b0 tx=0x7efc6400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.057+0000 7efc70ff9700 1 -- 172.21.15.112:0/1711373984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64007b00 con 0x7efc74069f40 2024-04-24T23:11:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.057+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc741992d0 con 0x7efc74069f40 2024-04-24T23:11:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.057+0000 7efc70ff9700 1 -- 172.21.15.112:0/1711373984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efc6400ce50 con 0x7efc74069f40 2024-04-24T23:11:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.057+0000 7efc70ff9700 1 -- 172.21.15.112:0/1711373984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc640186f0 con 0x7efc74069f40 2024-04-24T23:11:41.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.057+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc741a54e0 con 0x7efc74069f40 2024-04-24T23:11:41.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.058+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc74103960 con 0x7efc74069f40 2024-04-24T23:11:41.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.059+0000 7efc70ff9700 1 -- 172.21.15.112:0/1711373984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7efc64007ca0 con 0x7efc74069f40 2024-04-24T23:11:41.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.059+0000 7efc70ff9700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efc5c0a2c20 0x7efc5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:41.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.059+0000 7efc72ffd700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efc5c0a2c20 0x7efc5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:41.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.059+0000 7efc70ff9700 1 -- 172.21.15.112:0/1711373984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efc640cea70 con 0x7efc74069f40 2024-04-24T23:11:41.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.060+0000 7efc72ffd700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efc5c0a2c20 0x7efc5c0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc74078500 tx=0x7efc680073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:41.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.065+0000 7efc70ff9700 1 -- 172.21.15.112:0/1711373984 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc64014030 con 0x7efc74069f40 2024-04-24T23:11:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.259+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc741a5770 con 0x7efc74069f40 2024-04-24T23:11:41.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.260+0000 7efc70ff9700 1 -- 172.21.15.112:0/1711373984 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18515 (secure 0 0 0) 0x7efc6409be70 con 0x7efc74069f40 2024-04-24T23:11:41.262 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:11:41.262 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:11:38.636461+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efc5c0a2c20 msgr2=0x7efc5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efc5c0a2c20 0x7efc5c0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc74078500 tx=0x7efc680073d0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc74069f40 msgr2=0x7efc7419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc74069f40 0x7efc7419e680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efc6400c9b0 tx=0x7efc6400ccc0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 shutdown_connections 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efc5c0a2c20 0x7efc5c0a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc741082b0 0x7efc74198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc74069f40 0x7efc7419e680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 --2- 172.21.15.112:0/1711373984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7406a8e0 0x7efc7419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:41.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 >> 172.21.15.112:0/1711373984 conn(0x7efc740780e0 msgr2=0x7efc740ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:41.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 shutdown_connections 2024-04-24T23:11:41.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:41.262+0000 7efc79af2700 1 -- 172.21.15.112:0/1711373984 wait complete. 2024-04-24T23:11:41.971 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:41 smithi112 ceph-mon[29186]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:41.971 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1711373984' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:41.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:41 smithi187 ceph-mon[33292]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:41.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1711373984' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:42.001 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:11:42.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T23:11:42.067 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:41 smithi150 ceph-mon[32247]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:42.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1711373984' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:42.239 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.817+0000 7f99a68c4700 1 -- 172.21.15.112:0/2628632945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0108300 msgr2=0x7f99a01086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.817+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2628632945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0108300 0x7f99a01086e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9990009990 tx=0x7f9990009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.817+0000 7f99a68c4700 1 -- 172.21.15.112:0/2628632945 shutdown_connections 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.817+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2628632945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a0102c40 0x7f99a010f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.817+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2628632945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0108300 0x7f99a01086e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.817+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2628632945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a0102280 0x7f99a0102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.818+0000 7f99a68c4700 1 -- 172.21.15.112:0/2628632945 >> 172.21.15.112:0/2628632945 conn(0x7f99a00793e0 msgr2=0x7f99a00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.818+0000 7f99a68c4700 1 -- 172.21.15.112:0/2628632945 shutdown_connections 2024-04-24T23:11:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.818+0000 7f99a68c4700 1 -- 172.21.15.112:0/2628632945 wait complete. 2024-04-24T23:11:43.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.818+0000 7f99a68c4700 1 Processor -- start 2024-04-24T23:11:43.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.818+0000 7f99a68c4700 1 -- start start 2024-04-24T23:11:43.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.818+0000 7f99a68c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a0102280 0x7f99a019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:43.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f99a68c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0102c40 0x7f99a019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:43.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f99a68c4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a0108300 0x7f99a0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f99a68c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0112680 con 0x7f99a0102c40 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f99a68c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0112500 con 0x7f99a0102280 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f99a68c4700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0112800 con 0x7f99a0108300 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0102c40 0x7f99a019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0102c40 0x7f99a019ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47404/0 (socket says 172.21.15.112:47404) 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 -- 172.21.15.112:0/2158487839 learned_addr learned my addr 172.21.15.112:0/2158487839 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999ffff700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a0102280 0x7f99a019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f99a4e61700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a0108300 0x7f99a0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 -- 172.21.15.112:0/2158487839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a0108300 msgr2=0x7f99a0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a0108300 0x7f99a0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 -- 172.21.15.112:0/2158487839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a0102280 msgr2=0x7f99a019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a0102280 0x7f99a019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 -- 172.21.15.112:0/2158487839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a01990c0 con 0x7f99a0102c40 2024-04-24T23:11:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999ffff700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a0102280 0x7f99a019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999f7fe700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0102c40 0x7f99a019ec90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f999400d980 tx=0x7f999400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999d7fa700 1 -- 172.21.15.112:0/2158487839 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994017070 con 0x7f99a0102c40 2024-04-24T23:11:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a01993b0 con 0x7f99a0102c40 2024-04-24T23:11:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999d7fa700 1 -- 172.21.15.112:0/2158487839 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9994004d30 con 0x7f99a0102c40 2024-04-24T23:11:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f999d7fa700 1 -- 172.21.15.112:0/2158487839 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994005610 con 0x7f99a0102c40 2024-04-24T23:11:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.819+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a01a5620 con 0x7f99a0102c40 2024-04-24T23:11:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.820+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a0068f20 con 0x7f99a0102c40 2024-04-24T23:11:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.821+0000 7f999d7fa700 1 -- 172.21.15.112:0/2158487839 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9994004750 con 0x7f99a0102c40 2024-04-24T23:11:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.822+0000 7f999d7fa700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f99880a2c20 0x7f99880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.822+0000 7f999ffff700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f99880a2c20 0x7f99880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.822+0000 7f999d7fa700 1 -- 172.21.15.112:0/2158487839 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f99940ceb60 con 0x7f99a0102c40 2024-04-24T23:11:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.822+0000 7f999ffff700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f99880a2c20 0x7f99880a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9990006010 tx=0x7f999000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:43.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:43.827+0000 7f999d7fa700 1 -- 172.21.15.112:0/2158487839 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9994020020 con 0x7f99a0102c40 2024-04-24T23:11:44.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.023+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99a002db30 con 0x7f99a0102c40 2024-04-24T23:11:44.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.023+0000 7f999d7fa700 1 -- 172.21.15.112:0/2158487839 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f999409bf60 con 0x7f99a0102c40 2024-04-24T23:11:44.024 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f99880a2c20 msgr2=0x7f99880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f99880a2c20 0x7f99880a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9990006010 tx=0x7f999000b540 comp rx=0 tx=0).stop 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0102c40 msgr2=0x7f99a019ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0102c40 0x7f99a019ec90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f999400d980 tx=0x7f999400dd40 comp rx=0 tx=0).stop 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 shutdown_connections 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f99880a2c20 0x7f99880a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99a0102280 0x7f99a019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99a0108300 0x7f99a0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 --2- 172.21.15.112:0/2158487839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f99a0102c40 0x7f99a019ec90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 >> 172.21.15.112:0/2158487839 conn(0x7f99a00793e0 msgr2=0x7f99a0100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:44.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 shutdown_connections 2024-04-24T23:11:44.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:44.026+0000 7f99a68c4700 1 -- 172.21.15.112:0/2158487839 wait complete. 2024-04-24T23:11:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:43 smithi187 ceph-mon[33292]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:43 smithi112 ceph-mon[29186]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:43 smithi150 ceph-mon[32247]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:44.602 INFO:tasks.cephadm:Adding mds.a on smithi112 2024-04-24T23:11:44.602 INFO:tasks.cephadm:Adding mds.d on smithi112 2024-04-24T23:11:44.602 INFO:tasks.cephadm:Adding mds.g on smithi112 2024-04-24T23:11:44.602 INFO:tasks.cephadm:Adding mds.j on smithi112 2024-04-24T23:11:44.602 INFO:tasks.cephadm:Adding mds.b on smithi150 2024-04-24T23:11:44.602 INFO:tasks.cephadm:Adding mds.e on smithi150 2024-04-24T23:11:44.602 INFO:tasks.cephadm:Adding mds.h on smithi150 2024-04-24T23:11:44.602 INFO:tasks.cephadm:Adding mds.k on smithi150 2024-04-24T23:11:44.603 INFO:tasks.cephadm:Adding mds.c on smithi187 2024-04-24T23:11:44.603 INFO:tasks.cephadm:Adding mds.f on smithi187 2024-04-24T23:11:44.603 INFO:tasks.cephadm:Adding mds.i on smithi187 2024-04-24T23:11:44.603 INFO:tasks.cephadm:Adding mds.l on smithi187 2024-04-24T23:11:44.603 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l' 2024-04-24T23:11:44.854 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:11:44.976 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:44 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2158487839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:11:45.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:44 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2158487839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:11:45.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:44 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2158487839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:11:46.114 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:45 smithi187 ceph-mon[33292]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:45 smithi112 ceph-mon[29186]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:46.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:45 smithi150 ceph-mon[32247]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 -- 172.21.15.187:0/4293160781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 msgr2=0x7f6e801018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 --2- 172.21.15.187:0/4293160781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 0x7f6e801018b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009990 tx=0x7f6e6c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 -- 172.21.15.187:0/4293160781 shutdown_connections 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 --2- 172.21.15.187:0/4293160781 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e801027a0 0x7f6e80102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 --2- 172.21.15.187:0/4293160781 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e80101df0 0x7f6e801096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 --2- 172.21.15.187:0/4293160781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 0x7f6e801018b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 -- 172.21.15.187:0/4293160781 >> 172.21.15.187:0/4293160781 conn(0x7f6e800fbee0 msgr2=0x7f6e800fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 -- 172.21.15.187:0/4293160781 shutdown_connections 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 -- 172.21.15.187:0/4293160781 wait complete. 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 Processor -- start 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.485+0000 7f6e86c55700 1 -- start start 2024-04-24T23:11:46.489 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e86c55700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 0x7f6e801051f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e86c55700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e80101df0 0x7f6e80103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e86c55700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e801027a0 0x7f6e80103d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e86c55700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8010cf30 con 0x7f6e80101df0 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e86c55700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8010cdb0 con 0x7f6e801027a0 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e86c55700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8010d0b0 con 0x7f6e80101450 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 0x7f6e801051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 0x7f6e801051f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:59534/0 (socket says 172.21.15.187:59534) 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 -- 172.21.15.187:0/3860108152 learned_addr learned my addr 172.21.15.187:0/3860108152 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e7ffff700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e80101df0 0x7f6e80103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e851f2700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e801027a0 0x7f6e80103d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 -- 172.21.15.187:0/3860108152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e801027a0 msgr2=0x7f6e80103d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e801027a0 0x7f6e80103d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.490 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 -- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e80101df0 msgr2=0x7f6e80103840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e80101df0 0x7f6e80103840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 -- 172.21.15.187:0/3860108152 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e80104630 con 0x7f6e80101450 2024-04-24T23:11:46.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e849f1700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 0x7f6e801051f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c990 tx=0x7f6e7400cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:46.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e7dffb700 1 -- 172.21.15.187:0/3860108152 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74007940 con 0x7f6e80101450 2024-04-24T23:11:46.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e7dffb700 1 -- 172.21.15.187:0/3860108152 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6e7400f450 con 0x7f6e80101450 2024-04-24T23:11:46.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e801a2740 con 0x7f6e80101450 2024-04-24T23:11:46.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.486+0000 7f6e7dffb700 1 -- 172.21.15.187:0/3860108152 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74018640 con 0x7f6e80101450 2024-04-24T23:11:46.491 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.487+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e801a2c50 con 0x7f6e80101450 2024-04-24T23:11:46.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.488+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e80068f20 con 0x7f6e80101450 2024-04-24T23:11:46.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.489+0000 7f6e7dffb700 1 -- 172.21.15.187:0/3860108152 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6e7400f600 con 0x7f6e80101450 2024-04-24T23:11:46.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.489+0000 7f6e7dffb700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6e700a2c20 0x7f6e700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:46.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.489+0000 7f6e7dffb700 1 -- 172.21.15.187:0/3860108152 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6e740cea90 con 0x7f6e80101450 2024-04-24T23:11:46.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.489+0000 7f6e7ffff700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6e700a2c20 0x7f6e700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:46.492 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.490+0000 7f6e7ffff700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6e700a2c20 0x7f6e700a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0098e0 tx=0x7f6e6c003a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:46.495 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.494+0000 7f6e7dffb700 1 -- 172.21.15.187:0/3860108152 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6e74014030 con 0x7f6e80101450 2024-04-24T23:11:46.697 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.695+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}) -- 0x7f6e800fe010 con 0x7f6e700a2c20 2024-04-24T23:11:46.705 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.703+0000 7f6e7dffb700 1 -- 172.21.15.187:0/3860108152 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6e800fe010 con 0x7f6e700a2c20 2024-04-24T23:11:46.706 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mds.all update... 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6e700a2c20 msgr2=0x7f6e700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6e700a2c20 0x7f6e700a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0098e0 tx=0x7f6e6c003a70 comp rx=0 tx=0).stop 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 msgr2=0x7f6e801051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 0x7f6e801051f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c990 tx=0x7f6e7400cca0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 shutdown_connections 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6e700a2c20 0x7f6e700a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e801027a0 0x7f6e80103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e80101450 0x7f6e801051f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.708 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 --2- 172.21.15.187:0/3860108152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e80101df0 0x7f6e80103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:46.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 >> 172.21.15.187:0/3860108152 conn(0x7f6e800fbee0 msgr2=0x7f6e800fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:46.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 shutdown_connections 2024-04-24T23:11:46.709 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-24T23:11:46.706+0000 7f6e86c55700 1 -- 172.21.15.187:0/3860108152 wait complete. 2024-04-24T23:11:47.404 DEBUG:teuthology.orchestra.run.smithi112:mds.a> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.a.service 2024-04-24T23:11:47.406 DEBUG:teuthology.orchestra.run.smithi112:mds.d> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.d.service 2024-04-24T23:11:47.449 DEBUG:teuthology.orchestra.run.smithi112:mds.g> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.g.service 2024-04-24T23:11:47.493 DEBUG:teuthology.orchestra.run.smithi112:mds.j> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.j.service 2024-04-24T23:11:47.535 DEBUG:teuthology.orchestra.run.smithi150:mds.b> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.b.service 2024-04-24T23:11:47.538 DEBUG:teuthology.orchestra.run.smithi150:mds.e> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.e.service 2024-04-24T23:11:47.581 DEBUG:teuthology.orchestra.run.smithi150:mds.h> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.h.service 2024-04-24T23:11:47.624 DEBUG:teuthology.orchestra.run.smithi150:mds.k> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.k.service 2024-04-24T23:11:47.668 DEBUG:teuthology.orchestra.run.smithi187:mds.c> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.c.service 2024-04-24T23:11:47.671 DEBUG:teuthology.orchestra.run.smithi187:mds.f> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.f.service 2024-04-24T23:11:47.714 DEBUG:teuthology.orchestra.run.smithi187:mds.i> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.i.service 2024-04-24T23:11:47.756 DEBUG:teuthology.orchestra.run.smithi187:mds.l> sudo journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.l.service 2024-04-24T23:11:47.801 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-24T23:11:47.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:47.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:47.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:47.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:47 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- 172.21.15.112:0/1994331714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c101870 msgr2=0x7f4f4c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 --2- 172.21.15.112:0/1994331714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c101870 0x7f4f4c101cf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009900 tx=0x7f4f3c02a410 comp rx=0 tx=0).stop 2024-04-24T23:11:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- 172.21.15.112:0/1994331714 shutdown_connections 2024-04-24T23:11:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 --2- 172.21.15.112:0/1994331714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f4c102230 0x7f4f4c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 --2- 172.21.15.112:0/1994331714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c101870 0x7f4f4c101cf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 --2- 172.21.15.112:0/1994331714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f4c100680 0x7f4f4c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- 172.21.15.112:0/1994331714 >> 172.21.15.112:0/1994331714 conn(0x7f4f4c0fbbf0 msgr2=0x7f4f4c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:47.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- 172.21.15.112:0/1994331714 shutdown_connections 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- 172.21.15.112:0/1994331714 wait complete. 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 Processor -- start 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- start start 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c100680 0x7f4f4c10b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f4c101870 0x7f4f4c109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f4c102230 0x7f4f4c109b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c10d540 con 0x7f4f4c100680 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c10d3c0 con 0x7f4f4c102230 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.905+0000 7f4f543d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c10d6c0 con 0x7f4f4c101870 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c100680 0x7f4f4c10b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c100680 0x7f4f4c10b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47434/0 (socket says 172.21.15.112:47434) 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 -- 172.21.15.112:0/2263290775 learned_addr learned my addr 172.21.15.112:0/2263290775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:47.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f51970640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f4c101870 0x7f4f4c109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52972640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f4c102230 0x7f4f4c109b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 -- 172.21.15.112:0/2263290775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f4c101870 msgr2=0x7f4f4c109640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f4c101870 0x7f4f4c109640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 -- 172.21.15.112:0/2263290775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f4c102230 msgr2=0x7f4f4c109b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f4c102230 0x7f4f4c109b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 -- 172.21.15.112:0/2263290775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c10a410 con 0x7f4f4c100680 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52972640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f4c102230 0x7f4f4c109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f52171640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c100680 0x7f4f4c10b000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c970 tx=0x7f4f4800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f437fe640 1 -- 172.21.15.112:0/2263290775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004220 con 0x7f4f4c100680 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f437fe640 1 -- 172.21.15.112:0/2263290775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4f480043c0 con 0x7f4f4c100680 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c1a1c80 con 0x7f4f4c100680 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f437fe640 1 -- 172.21.15.112:0/2263290775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48004a00 con 0x7f4f4c100680 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.906+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c1a21a0 con 0x7f4f4c100680 2024-04-24T23:11:47.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.907+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c068560 con 0x7f4f4c100680 2024-04-24T23:11:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.908+0000 7f4f437fe640 1 -- 172.21.15.112:0/2263290775 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4f480079b0 con 0x7f4f4c100680 2024-04-24T23:11:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.909+0000 7f4f437fe640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f380a2c10 0x7f4f380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.909+0000 7f4f51970640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f380a2c10 0x7f4f380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:47.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.909+0000 7f4f437fe640 1 -- 172.21.15.112:0/2263290775 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4f480ce900 con 0x7f4f4c100680 2024-04-24T23:11:47.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.909+0000 7f4f51970640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f380a2c10 0x7f4f380a50d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c006010 tx=0x7f4f3c031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:47.914+0000 7f4f437fe640 1 -- 172.21.15.112:0/2263290775 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f48014030 con 0x7f4f4c100680 2024-04-24T23:11:47.984 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:47.985 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:47 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:48.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:48.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:48.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:47 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:48.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.092+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f4c062370 con 0x7f4f4c100680 2024-04-24T23:11:48.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.093+0000 7f4f437fe640 1 -- 172.21.15.112:0/2263290775 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18515 (secure 0 0 0) 0x7f4f4809bd00 con 0x7f4f4c100680 2024-04-24T23:11:48.095 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:11:48.095 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:11:38.636461+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:48.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f380a2c10 msgr2=0x7f4f380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f380a2c10 0x7f4f380a50d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c006010 tx=0x7f4f3c031640 comp rx=0 tx=0).stop 2024-04-24T23:11:48.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c100680 msgr2=0x7f4f4c10b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c100680 0x7f4f4c10b000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800c970 tx=0x7f4f4800ce30 comp rx=0 tx=0).stop 2024-04-24T23:11:48.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 shutdown_connections 2024-04-24T23:11:48.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f380a2c10 0x7f4f380a50d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f4c102230 0x7f4f4c109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f4c101870 0x7f4f4c109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 --2- 172.21.15.112:0/2263290775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f4c100680 0x7f4f4c10b000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 >> 172.21.15.112:0/2263290775 conn(0x7f4f4c0fbbf0 msgr2=0x7f4f4c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:48.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 shutdown_connections 2024-04-24T23:11:48.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.095+0000 7f4f543d5640 1 -- 172.21.15.112:0/2263290775 wait complete. 2024-04-24T23:11:48.108 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:11:48.108 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 -- 172.21.15.112:0/1002247725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588100680 msgr2=0x7f0588100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 --2- 172.21.15.112:0/1002247725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588100680 0x7f0588100aa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0578009900 tx=0x7f057802a410 comp rx=0 tx=0).stop 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 -- 172.21.15.112:0/1002247725 shutdown_connections 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 --2- 172.21.15.112:0/1002247725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0588102230 0x7f0588108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 --2- 172.21.15.112:0/1002247725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588101870 0x7f0588101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 --2- 172.21.15.112:0/1002247725 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588100680 0x7f0588100aa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 -- 172.21.15.112:0/1002247725 >> 172.21.15.112:0/1002247725 conn(0x7f05880fbbf0 msgr2=0x7f05880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 -- 172.21.15.112:0/1002247725 shutdown_connections 2024-04-24T23:11:48.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 -- 172.21.15.112:0/1002247725 wait complete. 2024-04-24T23:11:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 Processor -- start 2024-04-24T23:11:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.208+0000 7f058cb68640 1 -- start start 2024-04-24T23:11:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058cb68640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588100680 0x7f0588196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058cb68640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0588101870 0x7f0588197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058cb68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588102230 0x7f058819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058cb68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f058810c420 con 0x7f0588102230 2024-04-24T23:11:48.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058cb68640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f058810c2a0 con 0x7f0588100680 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058cb68640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f058810c5a0 con 0x7f0588101870 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588102230 0x7f058819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588102230 0x7f058819e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47454/0 (socket says 172.21.15.112:47454) 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 -- 172.21.15.112:0/3404506798 learned_addr learned my addr 172.21.15.112:0/3404506798 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058659c640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588100680 0x7f0588196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0585d9b640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0588101870 0x7f0588197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 -- 172.21.15.112:0/3404506798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0588101870 msgr2=0x7f0588197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0588101870 0x7f0588197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 -- 172.21.15.112:0/3404506798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588100680 msgr2=0x7f0588196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588100680 0x7f0588196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 -- 172.21.15.112:0/3404506798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058819e710 con 0x7f0588102230 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f0586d9d640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588102230 0x7f058819e0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0580009fd0 tx=0x7f058000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f05777fe640 1 -- 172.21.15.112:0/3404506798 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0580017070 con 0x7f0588102230 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058819ea00 con 0x7f0588102230 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f05777fe640 1 -- 172.21.15.112:0/3404506798 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0580004020 con 0x7f0588102230 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f05777fe640 1 -- 172.21.15.112:0/3404506798 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05800106c0 con 0x7f0588102230 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.209+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058819ef00 con 0x7f0588102230 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.210+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0588107a40 con 0x7f0588102230 2024-04-24T23:11:48.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.211+0000 7f05777fe640 1 -- 172.21.15.112:0/3404506798 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f05800041d0 con 0x7f0588102230 2024-04-24T23:11:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.212+0000 7f05777fe640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f05700a2b90 0x7f05700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.212+0000 7f058659c640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f05700a2b90 0x7f05700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.212+0000 7f05777fe640 1 -- 172.21.15.112:0/3404506798 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f05800ce990 con 0x7f0588102230 2024-04-24T23:11:48.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.212+0000 7f058659c640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f05700a2b90 0x7f05700a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0578006010 tx=0x7f0578005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:48.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.216+0000 7f05777fe640 1 -- 172.21.15.112:0/3404506798 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f058009bd90 con 0x7f0588102230 2024-04-24T23:11:48.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.390+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f058802d390 con 0x7f0588102230 2024-04-24T23:11:48.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.390+0000 7f05777fe640 1 -- 172.21.15.112:0/3404506798 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f058009b690 con 0x7f0588102230 2024-04-24T23:11:48.392 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:11:48.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f05700a2b90 msgr2=0x7f05700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f05700a2b90 0x7f05700a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0578006010 tx=0x7f0578005be0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588102230 msgr2=0x7f058819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588102230 0x7f058819e0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0580009fd0 tx=0x7f058000b260 comp rx=0 tx=0).stop 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 shutdown_connections 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f05700a2b90 0x7f05700a5050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0588102230 0x7f058819e0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0588101870 0x7f0588197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 --2- 172.21.15.112:0/3404506798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0588100680 0x7f0588196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 >> 172.21.15.112:0/3404506798 conn(0x7f05880fbbf0 msgr2=0x7f0588104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:48.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 shutdown_connections 2024-04-24T23:11:48.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.392+0000 7f058cb68640 1 -- 172.21.15.112:0/3404506798 wait complete. 2024-04-24T23:11:48.405 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:48.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 -- 172.21.15.112:0/1315290749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0600ffa50 msgr2=0x7fa0600ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 --2- 172.21.15.112:0/1315290749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0600ffa50 0x7fa0600ffe70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa054009930 tx=0x7fa05402a420 comp rx=0 tx=0).stop 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 -- 172.21.15.112:0/1315290749 shutdown_connections 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 --2- 172.21.15.112:0/1315290749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0601033e0 0x7fa060109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 --2- 172.21.15.112:0/1315290749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa060102a20 0x7fa060102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 --2- 172.21.15.112:0/1315290749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0600ffa50 0x7fa0600ffe70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 -- 172.21.15.112:0/1315290749 >> 172.21.15.112:0/1315290749 conn(0x7fa0600fb380 msgr2=0x7fa0600fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 -- 172.21.15.112:0/1315290749 shutdown_connections 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 -- 172.21.15.112:0/1315290749 wait complete. 2024-04-24T23:11:48.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 Processor -- start 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 -- start start 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0600ffa50 0x7fa060197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa060102a20 0x7fa0601981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.501+0000 7fa066d09640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0601033e0 0x7fa06019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa066d09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa06010d5e0 con 0x7fa060102a20 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa066d09640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa06010d460 con 0x7fa0601033e0 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa066d09640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa06010d760 con 0x7fa0600ffa50 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa060102a20 0x7fa0601981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa060102a20 0x7fa0601981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47476/0 (socket says 172.21.15.112:47476) 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 -- 172.21.15.112:0/1336246699 learned_addr learned my addr 172.21.15.112:0/1336246699 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:48.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa064aa5640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0600ffa50 0x7fa060197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa0652a6640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0601033e0 0x7fa06019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 -- 172.21.15.112:0/1336246699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0600ffa50 msgr2=0x7fa060197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0600ffa50 0x7fa060197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 -- 172.21.15.112:0/1336246699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0601033e0 msgr2=0x7fa06019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0601033e0 0x7fa06019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 -- 172.21.15.112:0/1336246699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06019f880 con 0x7fa060102a20 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa0652a6640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0601033e0 0x7fa06019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05ffff640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa060102a20 0x7fa0601981b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e9f0 tx=0x7fa04c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05dffb640 1 -- 172.21.15.112:0/1336246699 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00cd80 con 0x7fa060102a20 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06019fb70 con 0x7fa060102a20 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05dffb640 1 -- 172.21.15.112:0/1336246699 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa04c005520 con 0x7fa060102a20 2024-04-24T23:11:48.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa05dffb640 1 -- 172.21.15.112:0/1336246699 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c010690 con 0x7fa060102a20 2024-04-24T23:11:48.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.502+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601a0040 con 0x7fa060102a20 2024-04-24T23:11:48.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.503+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa060108b50 con 0x7fa060102a20 2024-04-24T23:11:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.504+0000 7fa05dffb640 1 -- 172.21.15.112:0/1336246699 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa04c0046c0 con 0x7fa060102a20 2024-04-24T23:11:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.505+0000 7fa05dffb640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0500a2b90 0x7fa0500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.505+0000 7fa064aa5640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0500a2b90 0x7fa0500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.505+0000 7fa05dffb640 1 -- 172.21.15.112:0/1336246699 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa04c0cf0f0 con 0x7fa060102a20 2024-04-24T23:11:48.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.505+0000 7fa064aa5640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0500a2b90 0x7fa0500a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa054005ab0 tx=0x7fa054031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:48.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.509+0000 7fa05dffb640 1 -- 172.21.15.112:0/1336246699 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa04c09c4f0 con 0x7fa060102a20 2024-04-24T23:11:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.722+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa06002d370 con 0x7fa060102a20 2024-04-24T23:11:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.722+0000 7fa05dffb640 1 -- 172.21.15.112:0/1336246699 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa04c09bdf0 con 0x7fa060102a20 2024-04-24T23:11:48.724 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:11:48.724 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-24T23:11:48.724 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 1 2024-04-24T23:11:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0500a2b90 msgr2=0x7fa0500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0500a2b90 0x7fa0500a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa054005ab0 tx=0x7fa054031640 comp rx=0 tx=0).stop 2024-04-24T23:11:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa060102a20 msgr2=0x7fa0601981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa060102a20 0x7fa0601981b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e9f0 tx=0x7fa04c00edf0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 shutdown_connections 2024-04-24T23:11:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa0500a2b90 0x7fa0500a5050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0601033e0 0x7fa06019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa060102a20 0x7fa0601981b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 --2- 172.21.15.112:0/1336246699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0600ffa50 0x7fa060197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 >> 172.21.15.112:0/1336246699 conn(0x7fa0600fb380 msgr2=0x7fa060105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 shutdown_connections 2024-04-24T23:11:48.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:48.724+0000 7fa066d09640 1 -- 172.21.15.112:0/1336246699 wait complete. 2024-04-24T23:11:48.962 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:48 smithi187 ceph-mon[33292]: Deploying daemon mds.l on smithi187 2024-04-24T23:11:48.962 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:48 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2263290775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:48.962 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:48 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3404506798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:11:49.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:48 smithi112 ceph-mon[29186]: Deploying daemon mds.l on smithi187 2024-04-24T23:11:49.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:48 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2263290775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:49.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:48 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3404506798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:11:49.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:48 smithi150 ceph-mon[32247]: Deploying daemon mds.l on smithi187 2024-04-24T23:11:49.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:48 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2263290775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:49.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:48 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3404506798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:11:49.644 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:49 smithi187 systemd[1]: Starting Ceph mds.l for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:11:49.908 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:49 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1336246699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:49.909 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:49 smithi187 ceph-mon[33292]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:49.909 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:49 smithi187 podman[62948]: 2024-04-24 23:11:49.545765851 +0000 UTC m=+0.018141196 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:11:49.909 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:49 smithi187 podman[62948]: 2024-04-24 23:11:49.663516886 +0000 UTC m=+0.135892203 container create 4e6c70c50489794861c7798d0cfe358101aa3ba04d6e99d8056e76c29f4d7e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:11:49.909 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:49 smithi187 podman[62948]: 2024-04-24 23:11:49.907898804 +0000 UTC m=+0.380274133 container init 4e6c70c50489794861c7798d0cfe358101aa3ba04d6e99d8056e76c29f4d7e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:11:50.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:49 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1336246699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:50.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:49 smithi112 ceph-mon[29186]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:50.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:49 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1336246699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:50.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:49 smithi150 ceph-mon[32247]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:50.236 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:49 smithi187 podman[62948]: 2024-04-24 23:11:49.910687366 +0000 UTC m=+0.383062683 container start 4e6c70c50489794861c7798d0cfe358101aa3ba04d6e99d8056e76c29f4d7e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T23:11:50.236 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:49 smithi187 ceph-mds[62967]: -- 172.21.15.187:0/3058970585 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56223f9e9a40 con 0x56223f9bc800 2024-04-24T23:11:50.236 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l[62963]: starting mds.l at 2024-04-24T23:11:50.236 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:50 smithi187 bash[62948]: 4e6c70c50489794861c7798d0cfe358101aa3ba04d6e99d8056e76c29f4d7e4d 2024-04-24T23:11:50.236 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:11:50 smithi187 systemd[1]: Started Ceph mds.l for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:11:50.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:50.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.846+0000 7fa28fe7d640 1 -- 172.21.15.112:0/2998233718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2881074c0 msgr2=0x7fa2881098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.846+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/2998233718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2881074c0 0x7fa2881098b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa280009900 tx=0x7fa28002a410 comp rx=0 tx=0).stop 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 -- 172.21.15.112:0/2998233718 shutdown_connections 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/2998233718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2881074c0 0x7fa2881098b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/2998233718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa288104b90 0x7fa288106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/2998233718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa288102260 0x7fa288104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 -- 172.21.15.112:0/2998233718 >> 172.21.15.112:0/2998233718 conn(0x7fa2880fbbf0 msgr2=0x7fa2880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 -- 172.21.15.112:0/2998233718 shutdown_connections 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 -- 172.21.15.112:0/2998233718 wait complete. 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 Processor -- start 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 -- start start 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa288102260 0x7fa288197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa288104b90 0x7fa2881980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:50.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2881074c0 0x7fa28819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d5b0 con 0x7fa288102260 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d430 con 0x7fa2881074c0 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28fe7d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa28810d730 con 0x7fa288104b90 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.847+0000 7fa28dc19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa288102260 0x7fa288197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28dc19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa288102260 0x7fa288197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42718/0 (socket says 172.21.15.112:42718) 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28dc19640 1 -- 172.21.15.112:0/367912413 learned_addr learned my addr 172.21.15.112:0/367912413 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28e41a640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2881074c0 0x7fa28819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28d418640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa288104b90 0x7fa2881980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28dc19640 1 -- 172.21.15.112:0/367912413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa288104b90 msgr2=0x7fa2881980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28dc19640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa288104b90 0x7fa2881980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28dc19640 1 -- 172.21.15.112:0/367912413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2881074c0 msgr2=0x7fa28819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28dc19640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2881074c0 0x7fa28819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28dc19640 1 -- 172.21.15.112:0/367912413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28819f7b0 con 0x7fa288102260 2024-04-24T23:11:50.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28d418640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa288104b90 0x7fa2881980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28dc19640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa288102260 0x7fa288197b90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa28400ba50 tx=0x7fa28400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa27effd640 1 -- 172.21.15.112:0/367912413 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284019070 con 0x7fa288102260 2024-04-24T23:11:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa27effd640 1 -- 172.21.15.112:0/367912413 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa28400cbe0 con 0x7fa288102260 2024-04-24T23:11:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa27effd640 1 -- 172.21.15.112:0/367912413 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284007770 con 0x7fa288102260 2024-04-24T23:11:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28819faa0 con 0x7fa288102260 2024-04-24T23:11:50.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.848+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28819ffa0 con 0x7fa288102260 2024-04-24T23:11:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.849+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2880fd6e0 con 0x7fa288102260 2024-04-24T23:11:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.850+0000 7fa27effd640 1 -- 172.21.15.112:0/367912413 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa28401c070 con 0x7fa288102260 2024-04-24T23:11:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.851+0000 7fa27effd640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa2740a2b90 0x7fa2740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.851+0000 7fa28d418640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa2740a2b90 0x7fa2740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.851+0000 7fa27effd640 1 -- 172.21.15.112:0/367912413 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa2840ce220 con 0x7fa288102260 2024-04-24T23:11:50.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.851+0000 7fa28d418640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa2740a2b90 0x7fa2740a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa278005d00 tx=0x7fa278005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:50.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:50.856+0000 7fa27effd640 1 -- 172.21.15.112:0/367912413 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa28409afd0 con 0x7fa288102260 2024-04-24T23:11:50.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] up:boot 2024-04-24T23:11:50.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: fsmap 1 up:standby 2024-04-24T23:11:50.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:11:50.972 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:50.973 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:50.973 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:50.973 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:50.973 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:50.973 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:50 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] up:boot 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: fsmap 1 up:standby 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:51.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:50 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.073+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa28802d170 con 0x7fa288102260 2024-04-24T23:11:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.073+0000 7fa27effd640 1 -- 172.21.15.112:0/367912413 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fa28409f2d0 con 0x7fa288102260 2024-04-24T23:11:51.075 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:11:51.075 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-24T23:11:51.075 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-04-24T23:11:51.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa2740a2b90 msgr2=0x7fa2740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:51.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa2740a2b90 0x7fa2740a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa278005d00 tx=0x7fa278005c70 comp rx=0 tx=0).stop 2024-04-24T23:11:51.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa288102260 msgr2=0x7fa288197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:51.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa288102260 0x7fa288197b90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa28400ba50 tx=0x7fa28400bf10 comp rx=0 tx=0).stop 2024-04-24T23:11:51.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 shutdown_connections 2024-04-24T23:11:51.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa2740a2b90 0x7fa2740a5050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:51.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa2881074c0 0x7fa28819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:51.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa288104b90 0x7fa2881980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:51.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 --2- 172.21.15.112:0/367912413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa288102260 0x7fa288197b90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:51.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 >> 172.21.15.112:0/367912413 conn(0x7fa2880fbbf0 msgr2=0x7fa288102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:51.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 shutdown_connections 2024-04-24T23:11:51.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:51.075+0000 7fa28fe7d640 1 -- 172.21.15.112:0/367912413 wait complete. 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] up:boot 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: fsmap 1 up:standby 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:50 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:51 smithi187 ceph-mon[33292]: Deploying daemon mds.k on smithi150 2024-04-24T23:11:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:51 smithi187 ceph-mon[33292]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:51 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/367912413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:52.236 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:51 smithi150 ceph-mon[32247]: Deploying daemon mds.k on smithi150 2024-04-24T23:11:52.236 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:51 smithi150 ceph-mon[32247]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:52.237 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:51 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/367912413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:51 smithi112 ceph-mon[29186]: Deploying daemon mds.k on smithi150 2024-04-24T23:11:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:51 smithi112 ceph-mon[29186]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:51 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/367912413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:52.589 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:11:52 smithi150 podman[61667]: 2024-04-24 23:11:52.468920183 +0000 UTC m=+0.017890599 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:11:52.589 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:11:52 smithi150 podman[61667]: 2024-04-24 23:11:52.588198709 +0000 UTC m=+0.137169100 container create d0e0e2b7963d0a8ab3b89f07e9ff3bed2944263c450a567ba66058b2192c48b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T23:11:52.860 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:11:52 smithi150 podman[61667]: 2024-04-24 23:11:52.832739732 +0000 UTC m=+0.381710132 container init d0e0e2b7963d0a8ab3b89f07e9ff3bed2944263c450a567ba66058b2192c48b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-24T23:11:52.860 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:11:52 smithi150 podman[61667]: 2024-04-24 23:11:52.836489949 +0000 UTC m=+0.385460347 container start d0e0e2b7963d0a8ab3b89f07e9ff3bed2944263c450a567ba66058b2192c48b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T23:11:53.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 -- 172.21.15.112:0/4069064621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 msgr2=0x7f345c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 --2- 172.21.15.112:0/4069064621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 0x7f345c100aa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f344c009900 tx=0x7f344c02a410 comp rx=0 tx=0).stop 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 -- 172.21.15.112:0/4069064621 shutdown_connections 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 --2- 172.21.15.112:0/4069064621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c102230 0x7f345c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 --2- 172.21.15.112:0/4069064621 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345c101870 0x7f345c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 --2- 172.21.15.112:0/4069064621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 0x7f345c100aa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 -- 172.21.15.112:0/4069064621 >> 172.21.15.112:0/4069064621 conn(0x7f345c0fbbf0 msgr2=0x7f345c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 -- 172.21.15.112:0/4069064621 shutdown_connections 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.189+0000 7f34632e7640 1 -- 172.21.15.112:0/4069064621 wait complete. 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f34632e7640 1 Processor -- start 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f34632e7640 1 -- start start 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f34632e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 0x7f345c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f34632e7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c101870 0x7f345c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f34632e7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345c102230 0x7f345c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f34632e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f345c10c420 con 0x7f345c100680 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f34632e7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f345c10c2a0 con 0x7f345c102230 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f34632e7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f345c10c5a0 con 0x7f345c101870 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461083640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 0x7f345c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461083640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 0x7f345c069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42742/0 (socket says 172.21.15.112:42742) 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461083640 1 -- 172.21.15.112:0/2063075696 learned_addr learned my addr 172.21.15.112:0/2063075696 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3460882640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c101870 0x7f345c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461884640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345c102230 0x7f345c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461083640 1 -- 172.21.15.112:0/2063075696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c101870 msgr2=0x7f345c19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461083640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c101870 0x7f345c19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461083640 1 -- 172.21.15.112:0/2063075696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345c102230 msgr2=0x7f345c19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461083640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345c102230 0x7f345c19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461083640 1 -- 172.21.15.112:0/2063075696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345c19f7b0 con 0x7f345c100680 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3460882640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c101870 0x7f345c19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.190+0000 7f3461884640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345c102230 0x7f345c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.191+0000 7f3461083640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 0x7f345c069440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f344c009230 tx=0x7f344c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.191+0000 7f34527fc640 1 -- 172.21.15.112:0/2063075696 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c03f030 con 0x7f345c100680 2024-04-24T23:11:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.191+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345c19f9e0 con 0x7f345c100680 2024-04-24T23:11:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.191+0000 7f34527fc640 1 -- 172.21.15.112:0/2063075696 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f344c003d00 con 0x7f345c100680 2024-04-24T23:11:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.191+0000 7f34527fc640 1 -- 172.21.15.112:0/2063075696 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c035070 con 0x7f345c100680 2024-04-24T23:11:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.191+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f345c19ff20 con 0x7f345c100680 2024-04-24T23:11:53.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.192+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f345c1079a0 con 0x7f345c100680 2024-04-24T23:11:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.194+0000 7f34527fc640 1 -- 172.21.15.112:0/2063075696 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f344c042030 con 0x7f345c100680 2024-04-24T23:11:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.194+0000 7f34527fc640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f34480a2b90 0x7f34480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.194+0000 7f34527fc640 1 -- 172.21.15.112:0/2063075696 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f344c0ef660 con 0x7f345c100680 2024-04-24T23:11:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.194+0000 7f3460882640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f34480a2b90 0x7f34480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.195+0000 7f3460882640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f34480a2b90 0x7f34480a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f345c0691b0 tx=0x7f3458005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:53.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.197+0000 7f34527fc640 1 -- 172.21.15.112:0/2063075696 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f344c0bca60 con 0x7f345c100680 2024-04-24T23:11:53.318 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:11:52 smithi150 ceph-mds[61686]: -- 172.21.15.150:0/1431799594 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f22c7bfa40 con 0x55f22c792800 2024-04-24T23:11:53.318 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:11:52 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k[61682]: starting mds.k at 2024-04-24T23:11:53.318 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:11:52 smithi150 bash[61667]: d0e0e2b7963d0a8ab3b89f07e9ff3bed2944263c450a567ba66058b2192c48b1 2024-04-24T23:11:53.318 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:11:52 smithi150 systemd[1]: Started Ceph mds.k for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:11:53.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.413+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f345c02d330 con 0x7f345c100680 2024-04-24T23:11:53.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.413+0000 7f34527fc640 1 -- 172.21.15.112:0/2063075696 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f344c0bc360 con 0x7f345c100680 2024-04-24T23:11:53.415 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:11:53.416 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:11:53.416 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.415+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f34480a2b90 msgr2=0x7f34480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f34480a2b90 0x7f34480a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f345c0691b0 tx=0x7f3458005f90 comp rx=0 tx=0).stop 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 msgr2=0x7f345c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 0x7f345c069440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f344c009230 tx=0x7f344c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 shutdown_connections 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f34480a2b90 0x7f34480a5050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f345c102230 0x7f345c19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c101870 0x7f345c19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 --2- 172.21.15.112:0/2063075696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f345c100680 0x7f345c069440 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:53.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 >> 172.21.15.112:0/2063075696 conn(0x7f345c0fbbf0 msgr2=0x7f345c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:53.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 shutdown_connections 2024-04-24T23:11:53.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:53.416+0000 7f34632e7640 1 -- 172.21.15.112:0/2063075696 wait complete. 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] up:boot 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: fsmap 2 up:standby 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: Deploying daemon mds.j on smithi112 2024-04-24T23:11:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:53 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2063075696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] up:boot 2024-04-24T23:11:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: fsmap 2 up:standby 2024-04-24T23:11:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:11:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:54.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:54.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:54.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: Deploying daemon mds.j on smithi112 2024-04-24T23:11:54.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2063075696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] up:boot 2024-04-24T23:11:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: fsmap 2 up:standby 2024-04-24T23:11:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:11:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: Deploying daemon mds.j on smithi112 2024-04-24T23:11:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:53 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2063075696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:55.426 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 systemd[1]: Starting Ceph mds.j for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:11:55.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 -- 172.21.15.112:0/2825618847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 msgr2=0x7f939c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 --2- 172.21.15.112:0/2825618847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 0x7f939c109bd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9394009ca0 tx=0x7f939402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 -- 172.21.15.112:0/2825618847 shutdown_connections 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 --2- 172.21.15.112:0/2825618847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 0x7f939c109bd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 --2- 172.21.15.112:0/2825618847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c102980 0x7f939c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 --2- 172.21.15.112:0/2825618847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939c101790 0x7f939c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 -- 172.21.15.112:0/2825618847 >> 172.21.15.112:0/2825618847 conn(0x7f939c0fcd20 msgr2=0x7f939c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 -- 172.21.15.112:0/2825618847 shutdown_connections 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.529+0000 7f93a2990640 1 -- 172.21.15.112:0/2825618847 wait complete. 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a2990640 1 Processor -- start 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a2990640 1 -- start start 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a2990640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c101790 0x7f939c197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a2990640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939c102980 0x7f939c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a2990640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 0x7f939c19f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a2990640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f939c10d640 con 0x7f939c103340 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a2990640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f939c10d4c0 con 0x7f939c101790 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a2990640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f939c10d7c0 con 0x7f939c102980 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a0f2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 0x7f939c19f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a0f2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 0x7f939c19f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42764/0 (socket says 172.21.15.112:42764) 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a0f2d640 1 -- 172.21.15.112:0/4291866423 learned_addr learned my addr 172.21.15.112:0/4291866423 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f939bfff640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c101790 0x7f939c197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f939b7fe640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939c102980 0x7f939c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a0f2d640 1 -- 172.21.15.112:0/4291866423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939c102980 msgr2=0x7f939c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a0f2d640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939c102980 0x7f939c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a0f2d640 1 -- 172.21.15.112:0/4291866423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c101790 msgr2=0x7f939c197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a0f2d640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c101790 0x7f939c197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.530+0000 7f93a0f2d640 1 -- 172.21.15.112:0/4291866423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c19f8d0 con 0x7f939c103340 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.531+0000 7f93a0f2d640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 0x7f939c19f230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9394009dc0 tx=0x7f939402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:55.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.531+0000 7f93997fa640 1 -- 172.21.15.112:0/4291866423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939402fe40 con 0x7f939c103340 2024-04-24T23:11:55.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.531+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c19fb60 con 0x7f939c103340 2024-04-24T23:11:55.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.531+0000 7f93997fa640 1 -- 172.21.15.112:0/4291866423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9394031df0 con 0x7f939c103340 2024-04-24T23:11:55.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.531+0000 7f93997fa640 1 -- 172.21.15.112:0/4291866423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394046ac0 con 0x7f939c103340 2024-04-24T23:11:55.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.531+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c1a0130 con 0x7f939c103340 2024-04-24T23:11:55.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.532+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939c108ab0 con 0x7f939c103340 2024-04-24T23:11:55.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.533+0000 7f93997fa640 1 -- 172.21.15.112:0/4291866423 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f93940305c0 con 0x7f939c103340 2024-04-24T23:11:55.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.533+0000 7f93997fa640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93840a2b90 0x7f93840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:55.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.533+0000 7f939bfff640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93840a2b90 0x7f93840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:55.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.534+0000 7f93997fa640 1 -- 172.21.15.112:0/4291866423 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f93940f0330 con 0x7f939c103340 2024-04-24T23:11:55.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.534+0000 7f939bfff640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93840a2b90 0x7f93840a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f939c198b60 tx=0x7f938c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:55.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.538+0000 7f93997fa640 1 -- 172.21.15.112:0/4291866423 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f93940bd730 con 0x7f939c103340 2024-04-24T23:11:55.682 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 podman[70358]: 2024-04-24 23:11:55.327802521 +0000 UTC m=+0.018109695 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:11:55.683 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 podman[70358]: 2024-04-24 23:11:55.453660704 +0000 UTC m=+0.143967861 container create b1feabf180444c65346d0759330c9c49d62a72a5906a2cbe900d4fc65c1d6795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T23:11:55.683 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 podman[70358]: 2024-04-24 23:11:55.681744247 +0000 UTC m=+0.372051404 container init b1feabf180444c65346d0759330c9c49d62a72a5906a2cbe900d4fc65c1d6795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T23:11:55.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.762+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f939c02d3c0 con 0x7f939c103340 2024-04-24T23:11:55.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.762+0000 7f93997fa640 1 -- 172.21.15.112:0/4291866423 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f93940bd030 con 0x7f939c103340 2024-04-24T23:11:55.765 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:11:55.765 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:11:55.765 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-04-24T23:11:55.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.764+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93840a2b90 msgr2=0x7f93840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.764+0000 7f93a2990640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93840a2b90 0x7f93840a5050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f939c198b60 tx=0x7f938c009110 comp rx=0 tx=0).stop 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.764+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 msgr2=0x7f939c19f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.764+0000 7f93a2990640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 0x7f939c19f230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9394009dc0 tx=0x7f939402f840 comp rx=0 tx=0).stop 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.765+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 shutdown_connections 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.765+0000 7f93a2990640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93840a2b90 0x7f93840a5050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.765+0000 7f93a2990640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f939c103340 0x7f939c19f230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.765+0000 7f93a2990640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939c102980 0x7f939c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.765+0000 7f93a2990640 1 --2- 172.21.15.112:0/4291866423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f939c101790 0x7f939c197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.765+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 >> 172.21.15.112:0/4291866423 conn(0x7f939c0fcd20 msgr2=0x7f939c105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.765+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 shutdown_connections 2024-04-24T23:11:55.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:55.765+0000 7f93a2990640 1 -- 172.21.15.112:0/4291866423 wait complete. 2024-04-24T23:11:55.958 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 podman[70358]: 2024-04-24 23:11:55.685291864 +0000 UTC m=+0.375599030 container start b1feabf180444c65346d0759330c9c49d62a72a5906a2cbe900d4fc65c1d6795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T23:11:55.958 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mds[70416]: -- 172.21.15.112:0/1887515681 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5568d5d57a40 con 0x5568d5d2a800 2024-04-24T23:11:55.958 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j[70412]: starting mds.j at 2024-04-24T23:11:55.958 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 bash[70358]: b1feabf180444c65346d0759330c9c49d62a72a5906a2cbe900d4fc65c1d6795 2024-04-24T23:11:55.958 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:11:55 smithi112 systemd[1]: Started Ceph mds.j for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:11:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:55 smithi187 ceph-mon[33292]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:55 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4291866423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:55 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mon[29186]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4291866423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:56.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:56.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:55 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:55 smithi150 ceph-mon[32247]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:55 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4291866423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:55 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:56 smithi112 ceph-mon[29186]: Deploying daemon mds.d on smithi112 2024-04-24T23:11:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:56 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.112:6834/4200698623,v1:172.21.15.112:6835/4200698623] up:boot 2024-04-24T23:11:57.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:56 smithi112 ceph-mon[29186]: fsmap 3 up:standby 2024-04-24T23:11:57.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:56 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:11:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:56 smithi187 ceph-mon[33292]: Deploying daemon mds.d on smithi112 2024-04-24T23:11:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:56 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.112:6834/4200698623,v1:172.21.15.112:6835/4200698623] up:boot 2024-04-24T23:11:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:56 smithi187 ceph-mon[33292]: fsmap 3 up:standby 2024-04-24T23:11:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:56 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:11:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:56 smithi150 ceph-mon[32247]: Deploying daemon mds.d on smithi112 2024-04-24T23:11:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:56 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.112:6834/4200698623,v1:172.21.15.112:6835/4200698623] up:boot 2024-04-24T23:11:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:56 smithi150 ceph-mon[32247]: fsmap 3 up:standby 2024-04-24T23:11:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:56 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-24T23:11:57.778 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:57.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.920+0000 7fbef4dc1640 1 -- 172.21.15.112:0/3723060382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0107460 msgr2=0x7fbef01098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.920+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/3723060382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0107460 0x7fbef01098b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009ca0 tx=0x7fbee802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.920+0000 7fbef4dc1640 1 -- 172.21.15.112:0/3723060382 shutdown_connections 2024-04-24T23:11:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.920+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/3723060382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0107460 0x7fbef01098b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.920+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/3723060382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbef0104b30 0x7fbef0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.920+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/3723060382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0102200 0x7fbef01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.920+0000 7fbef4dc1640 1 -- 172.21.15.112:0/3723060382 >> 172.21.15.112:0/3723060382 conn(0x7fbef00fbbf0 msgr2=0x7fbef00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.921+0000 7fbef4dc1640 1 -- 172.21.15.112:0/3723060382 shutdown_connections 2024-04-24T23:11:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.921+0000 7fbef4dc1640 1 -- 172.21.15.112:0/3723060382 wait complete. 2024-04-24T23:11:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.921+0000 7fbef4dc1640 1 Processor -- start 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbef4dc1640 1 -- start start 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbef4dc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0102200 0x7fbef0101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbef4dc1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbef0104b30 0x7fbef01001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbef4dc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0107460 0x7fbef01006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbef4dc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010d550 con 0x7fbef0102200 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbef4dc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010d3d0 con 0x7fbef0107460 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbef4dc1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010d6d0 con 0x7fbef0104b30 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0102200 0x7fbef0101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0102200 0x7fbef0101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42792/0 (socket says 172.21.15.112:42792) 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 -- 172.21.15.112:0/865410859 learned_addr learned my addr 172.21.15.112:0/865410859 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeedd9b640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbef0104b30 0x7fbef01001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeeed9d640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0107460 0x7fbef01006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:57.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 -- 172.21.15.112:0/865410859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbef0104b30 msgr2=0x7fbef01001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbef0104b30 0x7fbef01001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 -- 172.21.15.112:0/865410859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0107460 msgr2=0x7fbef01006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0107460 0x7fbef01006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 -- 172.21.15.112:0/865410859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef0100f80 con 0x7fbef0102200 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeeed9d640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0107460 0x7fbef01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.922+0000 7fbeee59c640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0102200 0x7fbef0101b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbee400c890 tx=0x7fbee400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.923+0000 7fbedf7fe640 1 -- 172.21.15.112:0/865410859 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4004220 con 0x7fbef0102200 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.923+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef01a1c30 con 0x7fbef0102200 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.923+0000 7fbedf7fe640 1 -- 172.21.15.112:0/865410859 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbee40043c0 con 0x7fbef0102200 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.923+0000 7fbedf7fe640 1 -- 172.21.15.112:0/865410859 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4004b10 con 0x7fbef0102200 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.923+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef01a21d0 con 0x7fbef0102200 2024-04-24T23:11:57.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.924+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef00fd6e0 con 0x7fbef0102200 2024-04-24T23:11:57.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.925+0000 7fbedf7fe640 1 -- 172.21.15.112:0/865410859 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbee40051b0 con 0x7fbef0102200 2024-04-24T23:11:57.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.925+0000 7fbedf7fe640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed80a2b90 0x7fbed80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:57.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.925+0000 7fbeedd9b640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed80a2b90 0x7fbed80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:57.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.925+0000 7fbedf7fe640 1 -- 172.21.15.112:0/865410859 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbee40cec40 con 0x7fbef0102200 2024-04-24T23:11:57.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.926+0000 7fbeedd9b640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed80a2b90 0x7fbed80a5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbef01018e0 tx=0x7fbee000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:57.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:57.930+0000 7fbedf7fe640 1 -- 172.21.15.112:0/865410859 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbee4014030 con 0x7fbef0102200 2024-04-24T23:11:58.051 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:57 smithi112 ceph-mon[29186]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.170+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef002d3b0 con 0x7fbef0102200 2024-04-24T23:11:58.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.170+0000 7fbedf7fe640 1 -- 172.21.15.112:0/865410859 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fbee409c040 con 0x7fbef0102200 2024-04-24T23:11:58.173 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:11:58.173 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:11:58.173 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 4 2024-04-24T23:11:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed80a2b90 msgr2=0x7fbed80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed80a2b90 0x7fbed80a5050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbef01018e0 tx=0x7fbee000b390 comp rx=0 tx=0).stop 2024-04-24T23:11:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0102200 msgr2=0x7fbef0101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0102200 0x7fbef0101b70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbee400c890 tx=0x7fbee400cd50 comp rx=0 tx=0).stop 2024-04-24T23:11:58.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 shutdown_connections 2024-04-24T23:11:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed80a2b90 0x7fbed80a5050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbef0107460 0x7fbef01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbef0104b30 0x7fbef01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 --2- 172.21.15.112:0/865410859 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0102200 0x7fbef0101b70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 >> 172.21.15.112:0/865410859 conn(0x7fbef00fbbf0 msgr2=0x7fbef0102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 shutdown_connections 2024-04-24T23:11:58.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:11:58.173+0000 7fbef4dc1640 1 -- 172.21.15.112:0/865410859 wait complete. 2024-04-24T23:11:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:57 smithi187 ceph-mon[33292]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:57 smithi150 ceph-mon[32247]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:11:58.779 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 24 23:11:58 smithi112 systemd[1]: Started Ceph mds.d for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:11:59.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:58 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/865410859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:59.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:59.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:59.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:58 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:59.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:58 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/865410859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:59.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:59.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:59.263 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:58 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:59.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:58 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/865410859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:59.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:11:59.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:11:59.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:11:59.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:58 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:00.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:59 smithi187 ceph-mon[33292]: Deploying daemon mds.e on smithi150 2024-04-24T23:12:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:59 smithi187 ceph-mon[33292]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:59 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.112:6836/3896751620,v1:172.21.15.112:6837/3896751620] up:boot 2024-04-24T23:12:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:59 smithi187 ceph-mon[33292]: fsmap 4 up:standby 2024-04-24T23:12:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:11:59 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:12:00.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:59 smithi112 ceph-mon[29186]: Deploying daemon mds.e on smithi150 2024-04-24T23:12:00.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:59 smithi112 ceph-mon[29186]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:00.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:59 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.112:6836/3896751620,v1:172.21.15.112:6837/3896751620] up:boot 2024-04-24T23:12:00.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:59 smithi112 ceph-mon[29186]: fsmap 4 up:standby 2024-04-24T23:12:00.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:11:59 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:12:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- 172.21.15.112:0/1343705202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4101870 msgr2=0x7f60d4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 --2- 172.21.15.112:0/1343705202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4101870 0x7f60d4101cf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009900 tx=0x7f60bc02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- 172.21.15.112:0/1343705202 shutdown_connections 2024-04-24T23:12:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 --2- 172.21.15.112:0/1343705202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d4102230 0x7f60d4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 --2- 172.21.15.112:0/1343705202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4101870 0x7f60d4101cf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 --2- 172.21.15.112:0/1343705202 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d4100680 0x7f60d4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- 172.21.15.112:0/1343705202 >> 172.21.15.112:0/1343705202 conn(0x7f60d40fbbf0 msgr2=0x7f60d40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- 172.21.15.112:0/1343705202 shutdown_connections 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- 172.21.15.112:0/1343705202 wait complete. 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 Processor -- start 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- start start 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4100680 0x7f60d4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d4101870 0x7f60d4198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d4102230 0x7f60d419f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60d410c430 con 0x7f60d4100680 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f60d410c2b0 con 0x7f60d4102230 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.299+0000 7f60daef5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f60d410c5b0 con 0x7f60d4101870 2024-04-24T23:12:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4100680 0x7f60d4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4100680 0x7f60d4197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42824/0 (socket says 172.21.15.112:42824) 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 -- 172.21.15.112:0/3395475659 learned_addr learned my addr 172.21.15.112:0/3395475659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d9492640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d4102230 0x7f60d419f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d3fff640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d4101870 0x7f60d4198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 -- 172.21.15.112:0/3395475659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d4101870 msgr2=0x7f60d4198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d4101870 0x7f60d4198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 -- 172.21.15.112:0/3395475659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d4102230 msgr2=0x7f60d419f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d4102230 0x7f60d419f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 -- 172.21.15.112:0/3395475659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d419f730 con 0x7f60d4100680 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d3fff640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d4101870 0x7f60d4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d9492640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d4102230 0x7f60d419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d8c91640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4100680 0x7f60d4197b10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00ccb0 tx=0x7f60cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d1ffb640 1 -- 172.21.15.112:0/3395475659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc003990 con 0x7f60d4100680 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d1ffb640 1 -- 172.21.15.112:0/3395475659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f60cc004520 con 0x7f60d4100680 2024-04-24T23:12:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60d1ffb640 1 -- 172.21.15.112:0/3395475659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc00f870 con 0x7f60d4100680 2024-04-24T23:12:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d419fa20 con 0x7f60d4100680 2024-04-24T23:12:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.300+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d419ff70 con 0x7f60d4100680 2024-04-24T23:12:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.301+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d41a01a0 con 0x7f60d4100680 2024-04-24T23:12:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.302+0000 7f60d1ffb640 1 -- 172.21.15.112:0/3395475659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f60cc0046d0 con 0x7f60d4100680 2024-04-24T23:12:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.303+0000 7f60d1ffb640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f60c00a2b90 0x7f60c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.303+0000 7f60d1ffb640 1 -- 172.21.15.112:0/3395475659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f60cc0cf020 con 0x7f60d4100680 2024-04-24T23:12:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.303+0000 7f60d3fff640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f60c00a2b90 0x7f60c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.303+0000 7f60d3fff640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f60c00a2b90 0x7f60c00a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f60bc000c00 tx=0x7f60bc009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:00.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.308+0000 7f60d1ffb640 1 -- 172.21.15.112:0/3395475659 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f60cc014030 con 0x7f60d4100680 2024-04-24T23:12:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:59 smithi150 ceph-mon[32247]: Deploying daemon mds.e on smithi150 2024-04-24T23:12:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:59 smithi150 ceph-mon[32247]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:59 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.112:6836/3896751620,v1:172.21.15.112:6837/3896751620] up:boot 2024-04-24T23:12:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:59 smithi150 ceph-mon[32247]: fsmap 4 up:standby 2024-04-24T23:12:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:11:59 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-24T23:12:00.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.522+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60d41a0370 con 0x7f60d4100680 2024-04-24T23:12:00.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.523+0000 7f60d1ffb640 1 -- 172.21.15.112:0/3395475659 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7f60cc09c420 con 0x7f60d4100680 2024-04-24T23:12:00.525 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:00.525 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-24T23:12:00.526 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 5 2024-04-24T23:12:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f60c00a2b90 msgr2=0x7f60c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f60c00a2b90 0x7f60c00a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f60bc000c00 tx=0x7f60bc009560 comp rx=0 tx=0).stop 2024-04-24T23:12:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4100680 msgr2=0x7f60d4197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4100680 0x7f60d4197b10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00ccb0 tx=0x7f60cc00a620 comp rx=0 tx=0).stop 2024-04-24T23:12:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 shutdown_connections 2024-04-24T23:12:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f60c00a2b90 0x7f60c00a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60d4102230 0x7f60d419f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60d4101870 0x7f60d4198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 --2- 172.21.15.112:0/3395475659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60d4100680 0x7f60d4197b10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:00.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 >> 172.21.15.112:0/3395475659 conn(0x7f60d40fbbf0 msgr2=0x7f60d40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:00.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 shutdown_connections 2024-04-24T23:12:00.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:00.525+0000 7f60daef5640 1 -- 172.21.15.112:0/3395475659 wait complete. 2024-04-24T23:12:00.979 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:00 smithi150 systemd[1]: Starting Ceph mds.e for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:12:01.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:00 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3395475659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:01.273 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:00 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3395475659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:01.273 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:01 smithi150 podman[61985]: 2024-04-24 23:12:00.937859788 +0000 UTC m=+0.018396692 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:12:01.274 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:01 smithi150 podman[61985]: 2024-04-24 23:12:01.036098232 +0000 UTC m=+0.116635118 container create a40ffab21ec45c71c150b950fa469a5251db1106fcee7314d31bf247f0aa6f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T23:12:01.274 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:01 smithi150 podman[61985]: 2024-04-24 23:12:01.272612471 +0000 UTC m=+0.353149357 container init a40ffab21ec45c71c150b950fa469a5251db1106fcee7314d31bf247f0aa6f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T23:12:01.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:00 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3395475659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:01.568 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:01 smithi150 podman[61985]: 2024-04-24 23:12:01.275376824 +0000 UTC m=+0.355913713 container start a40ffab21ec45c71c150b950fa469a5251db1106fcee7314d31bf247f0aa6f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T23:12:01.568 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-mds[62004]: -- 172.21.15.150:0/1607002228 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5edbe1a40 con 0x55d5edbb4800 2024-04-24T23:12:01.568 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e[62000]: starting mds.e at 2024-04-24T23:12:01.568 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:01 smithi150 bash[61985]: a40ffab21ec45c71c150b950fa469a5251db1106fcee7314d31bf247f0aa6f4f 2024-04-24T23:12:01.568 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:12:01 smithi150 systemd[1]: Started Ceph mds.e for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:12:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:01 smithi187 ceph-mon[33292]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:01 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:01 smithi112 ceph-mon[29186]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:01 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-mon[32247]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:01 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:02.539 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4151089476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc101880 msgr2=0x7fb1dc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4151089476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc101880 0x7fb1dc101d00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009900 tx=0x7fb1c802a3f0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4151089476 shutdown_connections 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4151089476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1dc102240 0x7fb1dc108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4151089476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc101880 0x7fb1dc101d00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4151089476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1dc100690 0x7fb1dc100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4151089476 >> 172.21.15.112:0/4151089476 conn(0x7fb1dc0fbc20 msgr2=0x7fb1dc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4151089476 shutdown_connections 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4151089476 wait complete. 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 Processor -- start 2024-04-24T23:12:02.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.648+0000 7fb1e2e88640 1 -- start start 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e2e88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc100690 0x7fb1dc104620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e2e88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1dc101880 0x7fb1dc102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e2e88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1dc102240 0x7fb1dc1031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e2e88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc10c440 con 0x7fb1dc100690 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e2e88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc10c2c0 con 0x7fb1dc101880 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e2e88640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc10c5c0 con 0x7fb1dc102240 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc100690 0x7fb1dc104620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc100690 0x7fb1dc104620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42322/0 (socket says 172.21.15.112:42322) 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 -- 172.21.15.112:0/4045228903 learned_addr learned my addr 172.21.15.112:0/4045228903 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1dbfff640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1dc101880 0x7fb1dc102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e1425640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1dc102240 0x7fb1dc1031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 -- 172.21.15.112:0/4045228903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1dc102240 msgr2=0x7fb1dc1031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1dc102240 0x7fb1dc1031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 -- 172.21.15.112:0/4045228903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1dc101880 msgr2=0x7fb1dc102c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1dc101880 0x7fb1dc102c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 -- 172.21.15.112:0/4045228903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc103a30 con 0x7fb1dc100690 2024-04-24T23:12:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e0c24640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc100690 0x7fb1dc104620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000c970 tx=0x7fb1d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/4045228903 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0007c10 con 0x7fb1dc100690 2024-04-24T23:12:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/4045228903 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb1d0007db0 con 0x7fb1dc100690 2024-04-24T23:12:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/4045228903 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d000fa40 con 0x7fb1dc100690 2024-04-24T23:12:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.649+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc1a1bb0 con 0x7fb1dc100690 2024-04-24T23:12:02.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.650+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc1a2120 con 0x7fb1dc100690 2024-04-24T23:12:02.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.651+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1dc0689b0 con 0x7fb1dc100690 2024-04-24T23:12:02.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.651+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/4045228903 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb1d000fbe0 con 0x7fb1dc100690 2024-04-24T23:12:02.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.652+0000 7fb1d9ffb640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1cc0a2c10 0x7fb1cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.652+0000 7fb1dbfff640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1cc0a2c10 0x7fb1cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.652+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/4045228903 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb1d00ceff0 con 0x7fb1dc100690 2024-04-24T23:12:02.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.653+0000 7fb1dbfff640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1cc0a2c10 0x7fb1cc0a50d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8000c00 tx=0x7fb1c8005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.657+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/4045228903 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1d009c480 con 0x7fb1dc100690 2024-04-24T23:12:02.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.872+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1dc1a24e0 con 0x7fb1dc100690 2024-04-24T23:12:02.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.872+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/4045228903 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fb1d009bd80 con 0x7fb1dc100690 2024-04-24T23:12:02.874 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:02.875 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-24T23:12:02.875 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 6 2024-04-24T23:12:02.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1cc0a2c10 msgr2=0x7fb1cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1cc0a2c10 0x7fb1cc0a50d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8000c00 tx=0x7fb1c8005be0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc100690 msgr2=0x7fb1dc104620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc100690 0x7fb1dc104620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000c970 tx=0x7fb1d000ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 shutdown_connections 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1cc0a2c10 0x7fb1cc0a50d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1dc102240 0x7fb1dc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1dc101880 0x7fb1dc102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 --2- 172.21.15.112:0/4045228903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1dc100690 0x7fb1dc104620 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 >> 172.21.15.112:0/4045228903 conn(0x7fb1dc0fbc20 msgr2=0x7fb1dc0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.874+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 shutdown_connections 2024-04-24T23:12:02.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:02.875+0000 7fb1e2e88640 1 -- 172.21.15.112:0/4045228903 wait complete. 2024-04-24T23:12:03.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:02 smithi112 ceph-mon[29186]: Deploying daemon mds.f on smithi187 2024-04-24T23:12:03.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:02 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6834/2076270934,v1:172.21.15.150:6835/2076270934] up:boot 2024-04-24T23:12:03.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:02 smithi112 ceph-mon[29186]: fsmap 5 up:standby 2024-04-24T23:12:03.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:02 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:12:03.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:02 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4045228903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:02 smithi150 ceph-mon[32247]: Deploying daemon mds.f on smithi187 2024-04-24T23:12:03.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:02 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6834/2076270934,v1:172.21.15.150:6835/2076270934] up:boot 2024-04-24T23:12:03.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:02 smithi150 ceph-mon[32247]: fsmap 5 up:standby 2024-04-24T23:12:03.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:02 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:12:03.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:02 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4045228903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.387 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:02 smithi187 ceph-mon[33292]: Deploying daemon mds.f on smithi187 2024-04-24T23:12:03.387 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:02 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6834/2076270934,v1:172.21.15.150:6835/2076270934] up:boot 2024-04-24T23:12:03.387 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:02 smithi187 ceph-mon[33292]: fsmap 5 up:standby 2024-04-24T23:12:03.388 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:02 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:12:03.388 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:02 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4045228903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.736 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:03 smithi187 systemd[1]: Starting Ceph mds.f for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:12:04.029 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:03 smithi187 ceph-mon[33292]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:04.029 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:03 smithi187 podman[63252]: 2024-04-24 23:12:03.681273585 +0000 UTC m=+0.018080656 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:12:04.030 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:03 smithi187 podman[63252]: 2024-04-24 23:12:03.79273685 +0000 UTC m=+0.129543906 container create ea1a12dc3cb2b06b8d2dd16b73fb056043723a993f227ab39e527f722b25f6bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:12:04.030 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:04 smithi187 podman[63252]: 2024-04-24 23:12:04.028754799 +0000 UTC m=+0.365561855 container init ea1a12dc3cb2b06b8d2dd16b73fb056043723a993f227ab39e527f722b25f6bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:12:04.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:03 smithi112 ceph-mon[29186]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:04.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:03 smithi150 ceph-mon[32247]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:04.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:04 smithi187 podman[63252]: 2024-04-24 23:12:04.031682317 +0000 UTC m=+0.368489380 container start ea1a12dc3cb2b06b8d2dd16b73fb056043723a993f227ab39e527f722b25f6bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-24T23:12:04.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:04 smithi187 ceph-mds[63271]: -- 172.21.15.187:0/3682160979 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556499e11a40 con 0x556499de5000 2024-04-24T23:12:04.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f[63267]: starting mds.f at 2024-04-24T23:12:04.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:04 smithi187 bash[63252]: ea1a12dc3cb2b06b8d2dd16b73fb056043723a993f227ab39e527f722b25f6bb 2024-04-24T23:12:04.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:12:04 smithi187 systemd[1]: Started Ceph mds.f for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:12:04.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.997+0000 7f61c94bd640 1 -- 172.21.15.112:0/3960995484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c41001b0 msgr2=0x7f61c4109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.997+0000 7f61c94bd640 1 --2- 172.21.15.112:0/3960995484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c41001b0 0x7f61c4109d60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009ca0 tx=0x7f61bc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.997+0000 7f61c94bd640 1 -- 172.21.15.112:0/3960995484 shutdown_connections 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 --2- 172.21.15.112:0/3960995484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c41001b0 0x7f61c4109d60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 --2- 172.21.15.112:0/3960995484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c40ff7f0 0x7f61c40ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 --2- 172.21.15.112:0/3960995484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c4102d80 0x7f61c41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 -- 172.21.15.112:0/3960995484 >> 172.21.15.112:0/3960995484 conn(0x7f61c40fb370 msgr2=0x7f61c40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 -- 172.21.15.112:0/3960995484 shutdown_connections 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 -- 172.21.15.112:0/3960995484 wait complete. 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 Processor -- start 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 -- start start 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c40ff7f0 0x7f61c4197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c41001b0 0x7f61c4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.998+0000 7f61c94bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c4102d80 0x7f61c419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c94bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61c410d540 con 0x7f61c40ff7f0 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c94bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f61c410d3c0 con 0x7f61c41001b0 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c94bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f61c410d6c0 con 0x7f61c4102d80 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c40ff7f0 0x7f61c4197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c40ff7f0 0x7f61c4197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42338/0 (socket says 172.21.15.112:42338) 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 -- 172.21.15.112:0/4055484938 learned_addr learned my addr 172.21.15.112:0/4055484938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c37fe640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c4102d80 0x7f61c419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c27fc640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c41001b0 0x7f61c4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 -- 172.21.15.112:0/4055484938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c4102d80 msgr2=0x7f61c419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c4102d80 0x7f61c419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 -- 172.21.15.112:0/4055484938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c41001b0 msgr2=0x7f61c4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c41001b0 0x7f61c4198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 -- 172.21.15.112:0/4055484938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c419f720 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c27fc640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c41001b0 0x7f61c4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c2ffd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c40ff7f0 0x7f61c4197b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61b400d950 tx=0x7f61b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61abfff640 1 -- 172.21.15.112:0/4055484938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4017020 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61abfff640 1 -- 172.21.15.112:0/4055484938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f61b4009c30 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c419fa10 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61abfff640 1 -- 172.21.15.112:0/4055484938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b400f7b0 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:04.999+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c419ff60 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.000+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c4067cd0 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.001+0000 7f61abfff640 1 -- 172.21.15.112:0/4055484938 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f61b401a020 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.002+0000 7f61abfff640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61ac0a2b90 0x7f61ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.002+0000 7f61c27fc640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61ac0a2b90 0x7f61ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.002+0000 7f61abfff640 1 -- 172.21.15.112:0/4055484938 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f61b40cee70 con 0x7f61c40ff7f0 2024-04-24T23:12:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.002+0000 7f61c27fc640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61ac0a2b90 0x7f61ac0a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f61c4199010 tx=0x7f61b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.006+0000 7f61abfff640 1 -- 172.21.15.112:0/4055484938 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f61b4021020 con 0x7f61c40ff7f0 2024-04-24T23:12:05.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.220+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61c4108b70 con 0x7f61c40ff7f0 2024-04-24T23:12:05.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.220+0000 7f61abfff640 1 -- 172.21.15.112:0/4055484938 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f61b4009de0 con 0x7f61c40ff7f0 2024-04-24T23:12:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:05.223 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-24T23:12:05.223 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-04-24T23:12:05.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61ac0a2b90 msgr2=0x7f61ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61ac0a2b90 0x7f61ac0a5050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f61c4199010 tx=0x7f61b8005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c40ff7f0 msgr2=0x7f61c4197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c40ff7f0 0x7f61c4197b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f61b400d950 tx=0x7f61b400de10 comp rx=0 tx=0).stop 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 shutdown_connections 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f61ac0a2b90 0x7f61ac0a5050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f61c4102d80 0x7f61c419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c41001b0 0x7f61c4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 --2- 172.21.15.112:0/4055484938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61c40ff7f0 0x7f61c4197b00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 >> 172.21.15.112:0/4055484938 conn(0x7f61c40fb370 msgr2=0x7f61c4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:05.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.222+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 shutdown_connections 2024-04-24T23:12:05.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:05.223+0000 7f61c94bd640 1 -- 172.21.15.112:0/4055484938 wait complete. 2024-04-24T23:12:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:05.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:05.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:05.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: Deploying daemon mds.c on smithi187 2024-04-24T23:12:05.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:05.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6834/1908796622,v1:172.21.15.187:6835/1908796622] up:boot 2024-04-24T23:12:05.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: fsmap 6 up:standby 2024-04-24T23:12:05.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: Deploying daemon mds.c on smithi187 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6834/1908796622,v1:172.21.15.187:6835/1908796622] up:boot 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: fsmap 6 up:standby 2024-04-24T23:12:05.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:12:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: Deploying daemon mds.c on smithi187 2024-04-24T23:12:05.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6834/1908796622,v1:172.21.15.187:6835/1908796622] up:boot 2024-04-24T23:12:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: fsmap 6 up:standby 2024-04-24T23:12:05.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:12:06.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:06 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4055484938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:06.486 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:06 smithi187 systemd[1]: Starting Ceph mds.c for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:12:06.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:06 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4055484938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:06.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:06 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4055484938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:06.942 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:06 smithi187 podman[63556]: 2024-04-24 23:12:06.489425422 +0000 UTC m=+0.018809555 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:12:06.942 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:06 smithi187 podman[63556]: 2024-04-24 23:12:06.68113566 +0000 UTC m=+0.210519764 container create 761366f0867225a49e85a1e3cac59f4068b017dd1fd3392a099b91b985c11c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:12:06.942 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:06 smithi187 podman[63556]: 2024-04-24 23:12:06.941428665 +0000 UTC m=+0.470812777 container init 761366f0867225a49e85a1e3cac59f4068b017dd1fd3392a099b91b985c11c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-24T23:12:07.195 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:06 smithi187 podman[63556]: 2024-04-24 23:12:06.943981299 +0000 UTC m=+0.473365404 container start 761366f0867225a49e85a1e3cac59f4068b017dd1fd3392a099b91b985c11c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:12:07.195 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:06 smithi187 ceph-mds[63574]: -- 172.21.15.187:0/3856108731 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b502f61a40 con 0x55b502f35000 2024-04-24T23:12:07.195 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c[63570]: starting mds.c at 2024-04-24T23:12:07.195 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:07 smithi187 bash[63556]: 761366f0867225a49e85a1e3cac59f4068b017dd1fd3392a099b91b985c11c78 2024-04-24T23:12:07.195 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:12:07 smithi187 systemd[1]: Started Ceph mds.c for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:12:07.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:07.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.340+0000 7f704d589640 1 -- 172.21.15.112:0/2809077733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481029a0 msgr2=0x7f7048102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.340+0000 7f704d589640 1 --2- 172.21.15.112:0/2809077733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481029a0 0x7f7048102e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f703c009900 tx=0x7f703c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 -- 172.21.15.112:0/2809077733 shutdown_connections 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 --2- 172.21.15.112:0/2809077733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7048103360 0x7f7048109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 --2- 172.21.15.112:0/2809077733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481029a0 0x7f7048102e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 --2- 172.21.15.112:0/2809077733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70481017b0 0x7f7048101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 -- 172.21.15.112:0/2809077733 >> 172.21.15.112:0/2809077733 conn(0x7f70480fcd20 msgr2=0x7f70480ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 -- 172.21.15.112:0/2809077733 shutdown_connections 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 -- 172.21.15.112:0/2809077733 wait complete. 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 Processor -- start 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 -- start start 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481017b0 0x7f70480754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70481029a0 0x7f7048073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:07.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7048103360 0x7f7048074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f704810d560 con 0x7f70481017b0 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.341+0000 7f704d589640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f704810d3e0 con 0x7f7048103360 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f704d589640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f704810d6e0 con 0x7f70481029a0 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481017b0 0x7f70480754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481017b0 0x7f70480754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42354/0 (socket says 172.21.15.112:42354) 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 -- 172.21.15.112:0/387145539 learned_addr learned my addr 172.21.15.112:0/387145539 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f70477fe640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7048103360 0x7f7048074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f70467fc640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70481029a0 0x7f7048073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 -- 172.21.15.112:0/387145539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70481029a0 msgr2=0x7f7048073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70481029a0 0x7f7048073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 -- 172.21.15.112:0/387145539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7048103360 msgr2=0x7f7048074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7048103360 0x7f7048074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 -- 172.21.15.112:0/387145539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7048074900 con 0x7f70481017b0 2024-04-24T23:12:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f7046ffd640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481017b0 0x7f70480754f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7038009fd0 tx=0x7f703800c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f702ffff640 1 -- 172.21.15.112:0/387145539 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038011470 con 0x7f70481017b0 2024-04-24T23:12:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f702ffff640 1 -- 172.21.15.112:0/387145539 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7038011b10 con 0x7f70481017b0 2024-04-24T23:12:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f702ffff640 1 -- 172.21.15.112:0/387145539 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703801acf0 con 0x7f70481017b0 2024-04-24T23:12:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70481a1d10 con 0x7f70481017b0 2024-04-24T23:12:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.342+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70481a21b0 con 0x7f70481017b0 2024-04-24T23:12:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.343+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7048108a70 con 0x7f70481017b0 2024-04-24T23:12:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.344+0000 7f702ffff640 1 -- 172.21.15.112:0/387145539 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7038011cc0 con 0x7f70481017b0 2024-04-24T23:12:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.345+0000 7f702ffff640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70300a2b90 0x7f70300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.345+0000 7f70467fc640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70300a2b90 0x7f70300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.345+0000 7f702ffff640 1 -- 172.21.15.112:0/387145539 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f70380cf1c0 con 0x7f70481017b0 2024-04-24T23:12:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.345+0000 7f70467fc640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70300a2b90 0x7f70300a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7048075260 tx=0x7f703c031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:07.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.350+0000 7f702ffff640 1 -- 172.21.15.112:0/387145539 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7038016030 con 0x7f70481017b0 2024-04-24T23:12:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:07 smithi187 ceph-mon[33292]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:07.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:07 smithi112 ceph-mon[29186]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:07.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:07.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:07.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.561+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f704802d170 con 0x7f70481017b0 2024-04-24T23:12:07.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.562+0000 7f702ffff640 1 -- 172.21.15.112:0/387145539 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f703809c5c0 con 0x7f70481017b0 2024-04-24T23:12:07.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:07.564 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:12:07.564 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-24T23:12:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70300a2b90 msgr2=0x7f70300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70300a2b90 0x7f70300a5050 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7048075260 tx=0x7f703c031640 comp rx=0 tx=0).stop 2024-04-24T23:12:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481017b0 msgr2=0x7f70480754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:07.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481017b0 0x7f70480754f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7038009fd0 tx=0x7f703800c270 comp rx=0 tx=0).stop 2024-04-24T23:12:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 shutdown_connections 2024-04-24T23:12:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70300a2b90 0x7f70300a5050 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7048103360 0x7f7048074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70481029a0 0x7f7048073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 --2- 172.21.15.112:0/387145539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70481017b0 0x7f70480754f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 >> 172.21.15.112:0/387145539 conn(0x7f70480fcd20 msgr2=0x7f7048105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 shutdown_connections 2024-04-24T23:12:07.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:07.564+0000 7f704d589640 1 -- 172.21.15.112:0/387145539 wait complete. 2024-04-24T23:12:07.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:07 smithi150 ceph-mon[32247]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:07.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:07.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:07.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:07.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:08.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:08 smithi187 ceph-mon[33292]: Deploying daemon mds.h on smithi150 2024-04-24T23:12:08.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:08 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6836/35756805,v1:172.21.15.187:6837/35756805] up:boot 2024-04-24T23:12:08.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:08 smithi187 ceph-mon[33292]: fsmap 7 up:standby 2024-04-24T23:12:08.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:08 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:12:08.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:08 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/387145539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:08.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:08 smithi112 ceph-mon[29186]: Deploying daemon mds.h on smithi150 2024-04-24T23:12:08.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:08 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6836/35756805,v1:172.21.15.187:6837/35756805] up:boot 2024-04-24T23:12:08.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:08 smithi112 ceph-mon[29186]: fsmap 7 up:standby 2024-04-24T23:12:08.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:08 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:12:08.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:08 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/387145539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:08.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:08 smithi150 ceph-mon[32247]: Deploying daemon mds.h on smithi150 2024-04-24T23:12:08.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:08 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6836/35756805,v1:172.21.15.187:6837/35756805] up:boot 2024-04-24T23:12:08.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:08 smithi150 ceph-mon[32247]: fsmap 7 up:standby 2024-04-24T23:12:08.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:08 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:12:08.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:08 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/387145539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:09.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:09 smithi187 ceph-mon[33292]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:09.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:09 smithi112 ceph-mon[29186]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:09.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:09 smithi150 ceph-mon[32247]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:09.568 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:09 smithi150 systemd[1]: Starting Ceph mds.h for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:12:09.578 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 -- 172.21.15.112:0/1568169914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014103480 msgr2=0x7f1014109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 --2- 172.21.15.112:0/1568169914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014103480 0x7f1014109d10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f100c009cd0 tx=0x7f100c02a8e0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 -- 172.21.15.112:0/1568169914 shutdown_connections 2024-04-24T23:12:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 --2- 172.21.15.112:0/1568169914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014103480 0x7f1014109d10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 --2- 172.21.15.112:0/1568169914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1014102ac0 0x7f1014102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 --2- 172.21.15.112:0/1568169914 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10140ffaf0 0x7f10140fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 -- 172.21.15.112:0/1568169914 >> 172.21.15.112:0/1568169914 conn(0x7f10140fb420 msgr2=0x7f10140fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 -- 172.21.15.112:0/1568169914 shutdown_connections 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.686+0000 7f101a8a5640 1 -- 172.21.15.112:0/1568169914 wait complete. 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f101a8a5640 1 Processor -- start 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f101a8a5640 1 -- start start 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f101a8a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10140ffaf0 0x7f1014197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f101a8a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014102ac0 0x7f1014198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f101a8a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1014103480 0x7f101419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f101a8a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f101410d600 con 0x7f1014102ac0 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f101a8a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f101410d480 con 0x7f10140ffaf0 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f101a8a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f101410d780 con 0x7f1014103480 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f10137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014102ac0 0x7f1014198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f10137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014102ac0 0x7f1014198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42378/0 (socket says 172.21.15.112:42378) 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f10137fe640 1 -- 172.21.15.112:0/3368272968 learned_addr learned my addr 172.21.15.112:0/3368272968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f1018e42640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1014103480 0x7f101419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:09.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.687+0000 7f1013fff640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10140ffaf0 0x7f1014197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10137fe640 1 -- 172.21.15.112:0/3368272968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1014103480 msgr2=0x7f101419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10137fe640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1014103480 0x7f101419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10137fe640 1 -- 172.21.15.112:0/3368272968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10140ffaf0 msgr2=0x7f1014197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10137fe640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10140ffaf0 0x7f1014197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10137fe640 1 -- 172.21.15.112:0/3368272968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101419f7e0 con 0x7f1014102ac0 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f1013fff640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10140ffaf0 0x7f1014197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10137fe640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014102ac0 0x7f1014198100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00e9f0 tx=0x7f0ffc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10117fa640 1 -- 172.21.15.112:0/3368272968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc00cd80 con 0x7f1014102ac0 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10117fa640 1 -- 172.21.15.112:0/3368272968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0ffc005520 con 0x7f1014102ac0 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101419fad0 con 0x7f1014102ac0 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f10117fa640 1 -- 172.21.15.112:0/3368272968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc010690 con 0x7f1014102ac0 2024-04-24T23:12:09.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.688+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101419ffd0 con 0x7f1014102ac0 2024-04-24T23:12:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.689+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1014108b90 con 0x7f1014102ac0 2024-04-24T23:12:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.690+0000 7f10117fa640 1 -- 172.21.15.112:0/3368272968 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0ffc0046c0 con 0x7f1014102ac0 2024-04-24T23:12:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.690+0000 7f10117fa640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f10000a2b90 0x7f10000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.691+0000 7f1013fff640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f10000a2b90 0x7f10000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.691+0000 7f10117fa640 1 -- 172.21.15.112:0/3368272968 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0ffc0cf100 con 0x7f1014102ac0 2024-04-24T23:12:09.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.691+0000 7f1013fff640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f10000a2b90 0x7f10000a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1014069c20 tx=0x7f1004005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:09.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.695+0000 7f10117fa640 1 -- 172.21.15.112:0/3368272968 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0ffc09c500 con 0x7f1014102ac0 2024-04-24T23:12:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.905+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1014062340 con 0x7f1014102ac0 2024-04-24T23:12:09.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.906+0000 7f10117fa640 1 -- 172.21.15.112:0/3368272968 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f0ffc09be00 con 0x7f1014102ac0 2024-04-24T23:12:09.908 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:09.908 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-24T23:12:09.908 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f10000a2b90 msgr2=0x7f10000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f10000a2b90 0x7f10000a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1014069c20 tx=0x7f1004005e70 comp rx=0 tx=0).stop 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014102ac0 msgr2=0x7f1014198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014102ac0 0x7f1014198100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00e9f0 tx=0x7f0ffc00edf0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 shutdown_connections 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f10000a2b90 0x7f10000a5050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1014103480 0x7f101419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1014102ac0 0x7f1014198100 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 --2- 172.21.15.112:0/3368272968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10140ffaf0 0x7f1014197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 >> 172.21.15.112:0/3368272968 conn(0x7f10140fb420 msgr2=0x7f1014105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:09.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 shutdown_connections 2024-04-24T23:12:09.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:09.908+0000 7f101a8a5640 1 -- 172.21.15.112:0/3368272968 wait complete. 2024-04-24T23:12:09.921 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:09 smithi150 podman[62300]: 2024-04-24 23:12:09.536465672 +0000 UTC m=+0.018424457 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:12:09.922 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:09 smithi150 podman[62300]: 2024-04-24 23:12:09.667692179 +0000 UTC m=+0.149650948 container create dfeb7077e80aed2d6324a6ba4cbc0b1302f77ad50f3e886d07738f7ce1defc6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T23:12:09.922 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:09 smithi150 podman[62300]: 2024-04-24 23:12:09.920241701 +0000 UTC m=+0.402200470 container init dfeb7077e80aed2d6324a6ba4cbc0b1302f77ad50f3e886d07738f7ce1defc6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T23:12:10.206 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:09 smithi150 podman[62300]: 2024-04-24 23:12:09.923168281 +0000 UTC m=+0.405127059 container start dfeb7077e80aed2d6324a6ba4cbc0b1302f77ad50f3e886d07738f7ce1defc6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:12:10.206 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:09 smithi150 ceph-mds[62319]: -- 172.21.15.150:0/434579971 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559955e79a40 con 0x559955e4d000 2024-04-24T23:12:10.206 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:09 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h[62315]: starting mds.h at 2024-04-24T23:12:10.206 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:10 smithi150 bash[62300]: dfeb7077e80aed2d6324a6ba4cbc0b1302f77ad50f3e886d07738f7ce1defc6c 2024-04-24T23:12:10.206 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:12:10 smithi150 systemd[1]: Started Ceph mds.h for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:12:10.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:10 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3368272968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:10.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:10.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:10.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:10 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:10.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:10 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3368272968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:10.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:10 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:10 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:10 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:10 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:10.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:10 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:10.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:10 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:10.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:10 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3368272968' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:10.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:10 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:10 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:10 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:10.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:10 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:10.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:10 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:10.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:10 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:11.479 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:11 smithi150 ceph-mon[32247]: Deploying daemon mds.i on smithi187 2024-04-24T23:12:11.479 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:11 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] up:boot 2024-04-24T23:12:11.479 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:11 smithi150 ceph-mon[32247]: fsmap 8 up:standby 2024-04-24T23:12:11.479 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:11 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:12:11.479 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:11 smithi150 ceph-mon[32247]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:11.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:11 smithi187 ceph-mon[33292]: Deploying daemon mds.i on smithi187 2024-04-24T23:12:11.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:11 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] up:boot 2024-04-24T23:12:11.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:11 smithi187 ceph-mon[33292]: fsmap 8 up:standby 2024-04-24T23:12:11.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:11 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:12:11.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:11 smithi187 ceph-mon[33292]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:11.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:11 smithi112 ceph-mon[29186]: Deploying daemon mds.i on smithi187 2024-04-24T23:12:11.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:11 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] up:boot 2024-04-24T23:12:11.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:11 smithi112 ceph-mon[29186]: fsmap 8 up:standby 2024-04-24T23:12:11.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:11 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:12:11.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:11 smithi112 ceph-mon[29186]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:11.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 -- 172.21.15.112:0/3063669289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 msgr2=0x7f80c0103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 --2- 172.21.15.112:0/3063669289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 0x7f80c0103dc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009900 tx=0x7f80b002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 -- 172.21.15.112:0/3063669289 shutdown_connections 2024-04-24T23:12:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 --2- 172.21.15.112:0/3063669289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80c0105550 0x7f80c010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 --2- 172.21.15.112:0/3063669289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c0104b90 0x7f80c0105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 --2- 172.21.15.112:0/3063669289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 0x7f80c0103dc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 -- 172.21.15.112:0/3063669289 >> 172.21.15.112:0/3063669289 conn(0x7f80c00fef30 msgr2=0x7f80c0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 -- 172.21.15.112:0/3063669289 shutdown_connections 2024-04-24T23:12:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.030+0000 7f80c627b640 1 -- 172.21.15.112:0/3063669289 wait complete. 2024-04-24T23:12:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c627b640 1 Processor -- start 2024-04-24T23:12:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c627b640 1 -- start start 2024-04-24T23:12:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c627b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 0x7f80c0199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:12.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c627b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80c0104b90 0x7f80c019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c627b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c0105550 0x7f80c01a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c627b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80c010f750 con 0x7f80c01039a0 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c627b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f80c010f5d0 con 0x7f80c0105550 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c627b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80c010f8d0 con 0x7f80c0104b90 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 0x7f80c0199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 0x7f80c0199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46610/0 (socket says 172.21.15.112:46610) 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80bffff640 1 -- 172.21.15.112:0/3656440359 learned_addr learned my addr 172.21.15.112:0/3656440359 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80bf7fe640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80c0104b90 0x7f80c019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:12.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.031+0000 7f80c4818640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c0105550 0x7f80c01a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bffff640 1 -- 172.21.15.112:0/3656440359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80c0104b90 msgr2=0x7f80c019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bffff640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80c0104b90 0x7f80c019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bffff640 1 -- 172.21.15.112:0/3656440359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c0105550 msgr2=0x7f80c01a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bffff640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c0105550 0x7f80c01a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bffff640 1 -- 172.21.15.112:0/3656440359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c01a19c0 con 0x7f80c01039a0 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bffff640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 0x7f80c0199db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009230 tx=0x7f80b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bd7fa640 1 -- 172.21.15.112:0/3656440359 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b003f030 con 0x7f80c01039a0 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bd7fa640 1 -- 172.21.15.112:0/3656440359 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f80b0003d00 con 0x7f80c01039a0 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80bd7fa640 1 -- 172.21.15.112:0/3656440359 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b0035070 con 0x7f80c01039a0 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c01a1c50 con 0x7f80c01039a0 2024-04-24T23:12:12.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.032+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c01a2140 con 0x7f80c01039a0 2024-04-24T23:12:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.033+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c010acc0 con 0x7f80c01039a0 2024-04-24T23:12:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.034+0000 7f80bd7fa640 1 -- 172.21.15.112:0/3656440359 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f80b0042030 con 0x7f80c01039a0 2024-04-24T23:12:12.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.034+0000 7f80bd7fa640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80a80a2b90 0x7f80a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.035+0000 7f80bf7fe640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80a80a2b90 0x7f80a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.035+0000 7f80bd7fa640 1 -- 172.21.15.112:0/3656440359 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f80b00ef070 con 0x7f80c01039a0 2024-04-24T23:12:12.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.035+0000 7f80bf7fe640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80a80a2b90 0x7f80a80a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009bd0 tx=0x7f80b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:12.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.039+0000 7f80bd7fa640 1 -- 172.21.15.112:0/3656440359 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f80b00bbe20 con 0x7f80c01039a0 2024-04-24T23:12:12.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.250+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80c002d3c0 con 0x7f80c01039a0 2024-04-24T23:12:12.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.251+0000 7f80bd7fa640 1 -- 172.21.15.112:0/3656440359 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f80b00c0120 con 0x7f80c01039a0 2024-04-24T23:12:12.253 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:12.253 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-24T23:12:12.253 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80a80a2b90 msgr2=0x7f80a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80a80a2b90 0x7f80a80a5050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009bd0 tx=0x7f80b4005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 msgr2=0x7f80c0199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 0x7f80c0199db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009230 tx=0x7f80b00042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 shutdown_connections 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80a80a2b90 0x7f80a80a5050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f80c0105550 0x7f80c01a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80c0104b90 0x7f80c019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 --2- 172.21.15.112:0/3656440359 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80c01039a0 0x7f80c0199db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 >> 172.21.15.112:0/3656440359 conn(0x7f80c00fef30 msgr2=0x7f80c0107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:12.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 shutdown_connections 2024-04-24T23:12:12.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:12.253+0000 7f80c627b640 1 -- 172.21.15.112:0/3656440359 wait complete. 2024-04-24T23:12:12.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:12 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3656440359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:12.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:12 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3656440359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:12.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:12 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3656440359' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:13.236 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:12:12 smithi187 systemd[1]: Started Ceph mds.i for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:12:14.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:14.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:14.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:14.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:14.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: Deploying daemon mds.g on smithi112 2024-04-24T23:12:14.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] up:boot 2024-04-24T23:12:14.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: fsmap 9 up:standby 2024-04-24T23:12:14.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:13 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:12:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:14.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: Deploying daemon mds.g on smithi112 2024-04-24T23:12:14.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] up:boot 2024-04-24T23:12:14.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: fsmap 9 up:standby 2024-04-24T23:12:14.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:13 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:12:14.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:14.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:14.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:14.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:14.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: Deploying daemon mds.g on smithi112 2024-04-24T23:12:14.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] up:boot 2024-04-24T23:12:14.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: fsmap 9 up:standby 2024-04-24T23:12:14.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:13 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:12:14.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 -- 172.21.15.112:0/1326570413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb00ff810 msgr2=0x7f1bb00ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 --2- 172.21.15.112:0/1326570413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb00ff810 0x7f1bb00ffc90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009900 tx=0x7f1ba402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 -- 172.21.15.112:0/1326570413 shutdown_connections 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 --2- 172.21.15.112:0/1326570413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb01001d0 0x7f1bb0109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 --2- 172.21.15.112:0/1326570413 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb00ff810 0x7f1bb00ffc90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 --2- 172.21.15.112:0/1326570413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bb0102da0 0x7f1bb01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 -- 172.21.15.112:0/1326570413 >> 172.21.15.112:0/1326570413 conn(0x7f1bb00fb3b0 msgr2=0x7f1bb00fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 -- 172.21.15.112:0/1326570413 shutdown_connections 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.367+0000 7f1bb664d640 1 -- 172.21.15.112:0/1326570413 wait complete. 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb664d640 1 Processor -- start 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb664d640 1 -- start start 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb664d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb00ff810 0x7f1bb0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:14.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb664d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb01001d0 0x7f1bb0198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb664d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bb0102da0 0x7f1bb019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb664d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb010d5a0 con 0x7f1bb01001d0 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb664d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb010d420 con 0x7f1bb00ff810 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb664d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb010d720 con 0x7f1bb0102da0 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb01001d0 0x7f1bb0198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb01001d0 0x7f1bb0198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46630/0 (socket says 172.21.15.112:46630) 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baf7fe640 1 -- 172.21.15.112:0/3904238343 learned_addr learned my addr 172.21.15.112:0/3904238343 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baffff640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb00ff810 0x7f1bb0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baf7fe640 1 -- 172.21.15.112:0/3904238343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bb0102da0 msgr2=0x7f1bb019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1bb4bea640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bb0102da0 0x7f1bb019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baf7fe640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bb0102da0 0x7f1bb019f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baf7fe640 1 -- 172.21.15.112:0/3904238343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb00ff810 msgr2=0x7f1bb0197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baf7fe640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb00ff810 0x7f1bb0197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.368+0000 7f1baf7fe640 1 -- 172.21.15.112:0/3904238343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb019f7f0 con 0x7f1bb01001d0 2024-04-24T23:12:14.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.369+0000 7f1baf7fe640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb01001d0 0x7f1bb0198110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009a20 tx=0x7f1ba4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:14.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.369+0000 7f1bad7fa640 1 -- 172.21.15.112:0/3904238343 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba403f030 con 0x7f1bb01001d0 2024-04-24T23:12:14.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.369+0000 7f1bad7fa640 1 -- 172.21.15.112:0/3904238343 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1ba4003d00 con 0x7f1bb01001d0 2024-04-24T23:12:14.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.369+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb019fa80 con 0x7f1bb01001d0 2024-04-24T23:12:14.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.369+0000 7f1bad7fa640 1 -- 172.21.15.112:0/3904238343 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4035070 con 0x7f1bb01001d0 2024-04-24T23:12:14.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.369+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb019ff70 con 0x7f1bb01001d0 2024-04-24T23:12:14.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.370+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb0108c60 con 0x7f1bb01001d0 2024-04-24T23:12:14.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.371+0000 7f1bad7fa640 1 -- 172.21.15.112:0/3904238343 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1ba4042030 con 0x7f1bb01001d0 2024-04-24T23:12:14.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.371+0000 7f1bad7fa640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1b980a2b90 0x7f1b980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:14.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.372+0000 7f1baffff640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1b980a2b90 0x7f1b980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:14.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.372+0000 7f1bad7fa640 1 -- 172.21.15.112:0/3904238343 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1ba40ef330 con 0x7f1bb01001d0 2024-04-24T23:12:14.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.372+0000 7f1baffff640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1b980a2b90 0x7f1b980a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0005d90 tx=0x7f1ba0005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:14.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.376+0000 7f1bad7fa640 1 -- 172.21.15.112:0/3904238343 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1ba40bc730 con 0x7f1bb01001d0 2024-04-24T23:12:14.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.591+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bb002d370 con 0x7f1bb01001d0 2024-04-24T23:12:14.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.592+0000 7f1bad7fa640 1 -- 172.21.15.112:0/3904238343 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7f1ba40bc030 con 0x7f1bb01001d0 2024-04-24T23:12:14.594 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:14.594 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:12:14.594 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 10 2024-04-24T23:12:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1b980a2b90 msgr2=0x7f1b980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1b980a2b90 0x7f1b980a5050 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0005d90 tx=0x7f1ba0005d20 comp rx=0 tx=0).stop 2024-04-24T23:12:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb01001d0 msgr2=0x7f1bb0198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb01001d0 0x7f1bb0198110 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009a20 tx=0x7f1ba4004430 comp rx=0 tx=0).stop 2024-04-24T23:12:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 shutdown_connections 2024-04-24T23:12:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1b980a2b90 0x7f1b980a5050 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1bb0102da0 0x7f1bb019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bb01001d0 0x7f1bb0198110 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 --2- 172.21.15.112:0/3904238343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1bb00ff810 0x7f1bb0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 >> 172.21.15.112:0/3904238343 conn(0x7f1bb00fb3b0 msgr2=0x7f1bb0105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 shutdown_connections 2024-04-24T23:12:14.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:14.594+0000 7f1bb664d640 1 -- 172.21.15.112:0/3904238343 wait complete. 2024-04-24T23:12:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:14 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3904238343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:15.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:14 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3904238343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:15.279 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 systemd[1]: Starting Ceph mds.g for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:12:15.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:14 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3904238343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:15.604 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 podman[71348]: 2024-04-24 23:12:15.251178483 +0000 UTC m=+0.018124507 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:12:15.604 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 podman[71348]: 2024-04-24 23:12:15.389264954 +0000 UTC m=+0.156210958 container create 40f6a20ca81b0fd09dedee7bc05c2fb849b0ca508ff0ff81ab626fc3c9d1386c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T23:12:15.943 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 podman[71348]: 2024-04-24 23:12:15.742065941 +0000 UTC m=+0.509011950 container init 40f6a20ca81b0fd09dedee7bc05c2fb849b0ca508ff0ff81ab626fc3c9d1386c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:12:15.943 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 podman[71348]: 2024-04-24 23:12:15.744823638 +0000 UTC m=+0.511769644 container start 40f6a20ca81b0fd09dedee7bc05c2fb849b0ca508ff0ff81ab626fc3c9d1386c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:12:15.943 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-mds[71367]: -- 172.21.15.112:0/57247340 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559091e19a40 con 0x559091dec800 2024-04-24T23:12:15.943 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g[71363]: starting mds.g at 2024-04-24T23:12:15.943 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 bash[71348]: 40f6a20ca81b0fd09dedee7bc05c2fb849b0ca508ff0ff81ab626fc3c9d1386c 2024-04-24T23:12:15.943 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:12:15 smithi112 systemd[1]: Started Ceph mds.g for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:12:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:15 smithi187 ceph-mon[33292]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:15 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-mon[29186]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:15 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:15 smithi150 ceph-mon[32247]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:15 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:16.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 -- 172.21.15.112:0/3585501996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80102230 msgr2=0x7f5d80108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3585501996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80102230 0x7f5d80108ac0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009900 tx=0x7f5d7802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 -- 172.21.15.112:0/3585501996 shutdown_connections 2024-04-24T23:12:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3585501996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80102230 0x7f5d80108ac0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3585501996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d80101870 0x7f5d80101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3585501996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80100680 0x7f5d80100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 -- 172.21.15.112:0/3585501996 >> 172.21.15.112:0/3585501996 conn(0x7f5d800fbbf0 msgr2=0x7f5d800fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:16.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 -- 172.21.15.112:0/3585501996 shutdown_connections 2024-04-24T23:12:16.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 -- 172.21.15.112:0/3585501996 wait complete. 2024-04-24T23:12:16.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.712+0000 7f5d8576a640 1 Processor -- start 2024-04-24T23:12:16.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d8576a640 1 -- start start 2024-04-24T23:12:16.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d8576a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d80100680 0x7f5d80196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d8576a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80101870 0x7f5d80197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d8576a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102230 0x7f5d8019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d8576a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8010c430 con 0x7f5d80101870 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d8576a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8010c2b0 con 0x7f5d80100680 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d8576a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8010c5b0 con 0x7f5d80102230 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80101870 0x7f5d80197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80101870 0x7f5d80197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46672/0 (socket says 172.21.15.112:46672) 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 -- 172.21.15.112:0/3975530983 learned_addr learned my addr 172.21.15.112:0/3975530983 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7f7fe640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102230 0x7f5d8019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7effd640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d80100680 0x7f5d80196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 -- 172.21.15.112:0/3975530983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102230 msgr2=0x7f5d8019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102230 0x7f5d8019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 -- 172.21.15.112:0/3975530983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d80100680 msgr2=0x7f5d80196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d80100680 0x7f5d80196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 -- 172.21.15.112:0/3975530983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8019e6c0 con 0x7f5d80101870 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7effd640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d80100680 0x7f5d80196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d7e7fc640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80101870 0x7f5d80197080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d950 tx=0x7f5d6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d77fff640 1 -- 172.21.15.112:0/3975530983 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d68017070 con 0x7f5d80101870 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d77fff640 1 -- 172.21.15.112:0/3975530983 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5d68009c30 con 0x7f5d80101870 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8019e9b0 con 0x7f5d80101870 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.713+0000 7f5d77fff640 1 -- 172.21.15.112:0/3975530983 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6800f740 con 0x7f5d80101870 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.714+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8019ef00 con 0x7f5d80101870 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.714+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d80107940 con 0x7f5d80101870 2024-04-24T23:12:16.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.715+0000 7f5d77fff640 1 -- 172.21.15.112:0/3975530983 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5d68010a80 con 0x7f5d80101870 2024-04-24T23:12:16.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.716+0000 7f5d77fff640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:16.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.716+0000 7f5d7effd640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:16.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.716+0000 7f5d77fff640 1 -- 172.21.15.112:0/3975530983 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5d680ce7f0 con 0x7f5d80101870 2024-04-24T23:12:16.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.717+0000 7f5d7effd640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005c40 tx=0x7f5d7000a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:16.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.721+0000 7f5d77fff640 1 -- 172.21.15.112:0/3975530983 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d68009de0 con 0x7f5d80101870 2024-04-24T23:12:16.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.948+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d8002d3c0 con 0x7f5d80101870 2024-04-24T23:12:16.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.948+0000 7f5d77fff640 1 -- 172.21.15.112:0/3975530983 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f5d6809bbf0 con 0x7f5d80101870 2024-04-24T23:12:16.950 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:16.950 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:12:16.951 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-24T23:12:16.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5d6c0a2b90 msgr2=0x7f5d6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d70005c40 tx=0x7f5d7000a430 comp rx=0 tx=0).stop 2024-04-24T23:12:16.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80101870 msgr2=0x7f5d80197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:16.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80101870 0x7f5d80197080 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5d6800d950 tx=0x7f5d6800de10 comp rx=0 tx=0).stop 2024-04-24T23:12:16.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 shutdown_connections 2024-04-24T23:12:16.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5d6c0a2b90 0x7f5d6c0a5050 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102230 0x7f5d8019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d80101870 0x7f5d80197080 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 --2- 172.21.15.112:0/3975530983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d80100680 0x7f5d80196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:16.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 >> 172.21.15.112:0/3975530983 conn(0x7f5d800fbbf0 msgr2=0x7f5d80104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:16.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 shutdown_connections 2024-04-24T23:12:16.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:16.950+0000 7f5d8576a640 1 -- 172.21.15.112:0/3975530983 wait complete. 2024-04-24T23:12:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:16 smithi187 ceph-mon[33292]: Deploying daemon mds.a on smithi112 2024-04-24T23:12:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:16 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.112:6838/2592086011,v1:172.21.15.112:6839/2592086011] up:boot 2024-04-24T23:12:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:16 smithi187 ceph-mon[33292]: fsmap 10 up:standby 2024-04-24T23:12:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:16 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:12:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:16 smithi112 ceph-mon[29186]: Deploying daemon mds.a on smithi112 2024-04-24T23:12:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:16 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.112:6838/2592086011,v1:172.21.15.112:6839/2592086011] up:boot 2024-04-24T23:12:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:16 smithi112 ceph-mon[29186]: fsmap 10 up:standby 2024-04-24T23:12:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:16 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:12:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:16 smithi150 ceph-mon[32247]: Deploying daemon mds.a on smithi112 2024-04-24T23:12:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:16 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.112:6838/2592086011,v1:172.21.15.112:6839/2592086011] up:boot 2024-04-24T23:12:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:16 smithi150 ceph-mon[32247]: fsmap 10 up:standby 2024-04-24T23:12:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:16 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:12:18.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:17 smithi112 ceph-mon[29186]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:18.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:17 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3975530983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:18.050 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:17 smithi112 systemd[1]: Starting Ceph mds.a for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:12:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:17 smithi187 ceph-mon[33292]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:17 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3975530983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:17 smithi150 ceph-mon[32247]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:17 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3975530983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:18.340 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:18 smithi112 podman[71698]: 2024-04-24 23:12:18.112468287 +0000 UTC m=+0.018437078 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:12:18.340 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:18 smithi112 podman[71698]: 2024-04-24 23:12:18.227812938 +0000 UTC m=+0.133781719 container create 6785c8969dea7c4c0901a5323ca88af5f1f846935d0613a193fc17cb5e68bb37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T23:12:18.779 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:18 smithi112 podman[71698]: 2024-04-24 23:12:18.455724923 +0000 UTC m=+0.361693705 container init 6785c8969dea7c4c0901a5323ca88af5f1f846935d0613a193fc17cb5e68bb37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-24T23:12:18.779 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:18 smithi112 podman[71698]: 2024-04-24 23:12:18.458621142 +0000 UTC m=+0.364589924 container start 6785c8969dea7c4c0901a5323ca88af5f1f846935d0613a193fc17cb5e68bb37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:12:18.779 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:18 smithi112 ceph-mds[71727]: -- 172.21.15.112:0/1956312591 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633730cfa40 con 0x5633730a2800 2024-04-24T23:12:18.779 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:18 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a[71721]: starting mds.a at 2024-04-24T23:12:18.779 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:18 smithi112 bash[71698]: 6785c8969dea7c4c0901a5323ca88af5f1f846935d0613a193fc17cb5e68bb37 2024-04-24T23:12:18.779 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:12:18 smithi112 systemd[1]: Started Ceph mds.a for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:12:18.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:19.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 -- 172.21.15.112:0/3843741278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 msgr2=0x7f4228106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 --2- 172.21.15.112:0/3843741278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 0x7f4228106f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f421c009900 tx=0x7f421c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:19.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 -- 172.21.15.112:0/3843741278 shutdown_connections 2024-04-24T23:12:19.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 --2- 172.21.15.112:0/3843741278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228107460 0x7f42281098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 --2- 172.21.15.112:0/3843741278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 0x7f4228106f20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 --2- 172.21.15.112:0/3843741278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4228102200 0x7f42281045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 -- 172.21.15.112:0/3843741278 >> 172.21.15.112:0/3843741278 conn(0x7f42280fbbf0 msgr2=0x7f42280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 -- 172.21.15.112:0/3843741278 shutdown_connections 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.075+0000 7f422d177640 1 -- 172.21.15.112:0/3843741278 wait complete. 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.076+0000 7f422d177640 1 Processor -- start 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.076+0000 7f422d177640 1 -- start start 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.076+0000 7f422d177640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4228102200 0x7f4228197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.076+0000 7f422d177640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 0x7f42281980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.076+0000 7f422d177640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228107460 0x7f422819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.076+0000 7f422d177640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f422810d530 con 0x7f4228104b30 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.076+0000 7f422d177640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f422810d3b0 con 0x7f4228107460 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.076+0000 7f422d177640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f422810d6b0 con 0x7f4228102200 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 0x7f42281980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:19.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 0x7f42281980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46712/0 (socket says 172.21.15.112:46712) 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 -- 172.21.15.112:0/1043419872 learned_addr learned my addr 172.21.15.112:0/1043419872 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422759e640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228107460 0x7f422819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f4226d9d640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4228102200 0x7f4228197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 -- 172.21.15.112:0/1043419872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4228102200 msgr2=0x7f4228197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4228102200 0x7f4228197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 -- 172.21.15.112:0/1043419872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228107460 msgr2=0x7f422819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228107460 0x7f422819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 -- 172.21.15.112:0/1043419872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422819f7c0 con 0x7f4228104b30 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422659c640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 0x7f42281980e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f421c009230 tx=0x7f421c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f420ffff640 1 -- 172.21.15.112:0/1043419872 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c03f030 con 0x7f4228104b30 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422819fa50 con 0x7f4228104b30 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f420ffff640 1 -- 172.21.15.112:0/1043419872 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f421c003d00 con 0x7f4228104b30 2024-04-24T23:12:19.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f420ffff640 1 -- 172.21.15.112:0/1043419872 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c035070 con 0x7f4228104b30 2024-04-24T23:12:19.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.077+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422819ff90 con 0x7f4228104b30 2024-04-24T23:12:19.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.078+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42280fd6e0 con 0x7f4228104b30 2024-04-24T23:12:19.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.080+0000 7f420ffff640 1 -- 172.21.15.112:0/1043419872 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f421c042030 con 0x7f4228104b30 2024-04-24T23:12:19.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.080+0000 7f420ffff640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f42100a2b90 0x7f42100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:19.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.080+0000 7f4226d9d640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f42100a2b90 0x7f42100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:19.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.080+0000 7f420ffff640 1 -- 172.21.15.112:0/1043419872 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f421c0ef490 con 0x7f4228104b30 2024-04-24T23:12:19.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.081+0000 7f4226d9d640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f42100a2b90 0x7f42100a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4218005fd0 tx=0x7f4218009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:19.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.084+0000 7f420ffff640 1 -- 172.21.15.112:0/1043419872 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f421c0bc240 con 0x7f4228104b30 2024-04-24T23:12:19.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.295+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f422802d3c0 con 0x7f4228104b30 2024-04-24T23:12:19.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.296+0000 7f420ffff640 1 -- 172.21.15.112:0/1043419872 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7f421c0c0540 con 0x7f4228104b30 2024-04-24T23:12:19.298 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:19.299 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:12:19.299 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-04-24T23:12:19.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f42100a2b90 msgr2=0x7f42100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f42100a2b90 0x7f42100a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4218005fd0 tx=0x7f4218009680 comp rx=0 tx=0).stop 2024-04-24T23:12:19.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 msgr2=0x7f42281980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:19.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 0x7f42281980e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f421c009230 tx=0x7f421c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 shutdown_connections 2024-04-24T23:12:19.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f42100a2b90 0x7f42100a5050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4228107460 0x7f422819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4228104b30 0x7f42281980e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 --2- 172.21.15.112:0/1043419872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4228102200 0x7f4228197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:19.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 >> 172.21.15.112:0/1043419872 conn(0x7f42280fbbf0 msgr2=0x7f4228102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:19.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 shutdown_connections 2024-04-24T23:12:19.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:19.299+0000 7f422d177640 1 -- 172.21.15.112:0/1043419872 wait complete. 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: Deploying daemon mds.b on smithi150 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:19.818 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.112:6840/3500837168,v1:172.21.15.112:6841/3500837168] up:boot 2024-04-24T23:12:19.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: fsmap 11 up:standby 2024-04-24T23:12:19.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:12:19.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:19 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1043419872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:19.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:19.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:19.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:19.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:19.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:19.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:19.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: Deploying daemon mds.b on smithi150 2024-04-24T23:12:19.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:19.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.112:6840/3500837168,v1:172.21.15.112:6841/3500837168] up:boot 2024-04-24T23:12:19.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: fsmap 11 up:standby 2024-04-24T23:12:19.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:12:19.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:19 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1043419872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:12:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:12:20.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:20.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: Deploying daemon mds.b on smithi150 2024-04-24T23:12:20.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:20.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.112:6840/3500837168,v1:172.21.15.112:6841/3500837168] up:boot 2024-04-24T23:12:20.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: fsmap 11 up:standby 2024-04-24T23:12:20.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:12:20.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:19 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1043419872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:21.068 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:20 smithi150 systemd[1]: Starting Ceph mds.b for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:12:21.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:21.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.422+0000 7ffadc189640 1 -- 172.21.15.112:0/4125602756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 msgr2=0x7ffad4102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:21.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.422+0000 7ffadc189640 1 --2- 172.21.15.112:0/4125602756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 0x7ffad4102e00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:21.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 -- 172.21.15.112:0/4125602756 shutdown_connections 2024-04-24T23:12:21.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 --2- 172.21.15.112:0/4125602756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad4103340 0x7ffad4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 --2- 172.21.15.112:0/4125602756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 0x7ffad4102e00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 --2- 172.21.15.112:0/4125602756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4101790 0x7ffad4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 -- 172.21.15.112:0/4125602756 >> 172.21.15.112:0/4125602756 conn(0x7ffad40fcd40 msgr2=0x7ffad40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:21.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 -- 172.21.15.112:0/4125602756 shutdown_connections 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 -- 172.21.15.112:0/4125602756 wait complete. 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 Processor -- start 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 -- start start 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4101790 0x7ffad4197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 0x7ffad41980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad4103340 0x7ffad419f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410d540 con 0x7ffad4102980 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410d3c0 con 0x7ffad4103340 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffadc189640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410d6c0 con 0x7ffad4101790 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffad9724640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 0x7ffad41980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:21.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffad9724640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 0x7ffad41980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41364/0 (socket says 172.21.15.112:41364) 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffad9724640 1 -- 172.21.15.112:0/2973059069 learned_addr learned my addr 172.21.15.112:0/2973059069 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.423+0000 7ffada726640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad4103340 0x7ffad419f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffad9f25640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4101790 0x7ffad4197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffad9724640 1 -- 172.21.15.112:0/2973059069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4101790 msgr2=0x7ffad4197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffad9724640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4101790 0x7ffad4197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffad9724640 1 -- 172.21.15.112:0/2973059069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad4103340 msgr2=0x7ffad419f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffad9724640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad4103340 0x7ffad419f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffad9724640 1 -- 172.21.15.112:0/2973059069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad419f7b0 con 0x7ffad4102980 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffad9f25640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4101790 0x7ffad4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffad9724640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 0x7ffad41980d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009a20 tx=0x7ffac4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffacaffd640 1 -- 172.21.15.112:0/2973059069 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac403f030 con 0x7ffad4102980 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffacaffd640 1 -- 172.21.15.112:0/2973059069 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffac4003d00 con 0x7ffad4102980 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad419fa40 con 0x7ffad4102980 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffacaffd640 1 -- 172.21.15.112:0/2973059069 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4035070 con 0x7ffad4102980 2024-04-24T23:12:21.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.424+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad419ff80 con 0x7ffad4102980 2024-04-24T23:12:21.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.425+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad4108ab0 con 0x7ffad4102980 2024-04-24T23:12:21.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.426+0000 7ffacaffd640 1 -- 172.21.15.112:0/2973059069 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ffac4042030 con 0x7ffad4102980 2024-04-24T23:12:21.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.427+0000 7ffacaffd640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffac00a2c10 0x7ffac00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:21.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.427+0000 7ffad9f25640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffac00a2c10 0x7ffac00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:21.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.427+0000 7ffacaffd640 1 -- 172.21.15.112:0/2973059069 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffac40ef1d0 con 0x7ffad4102980 2024-04-24T23:12:21.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.427+0000 7ffad9f25640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffac00a2c10 0x7ffac00a50d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffad00058c0 tx=0x7ffad0005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:21.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.431+0000 7ffacaffd640 1 -- 172.21.15.112:0/2973059069 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffac40bc5d0 con 0x7ffad4102980 2024-04-24T23:12:21.437 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:21 smithi150 podman[62616]: 2024-04-24 23:12:21.0883607 +0000 UTC m=+0.018392892 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:12:21.438 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:21 smithi150 podman[62616]: 2024-04-24 23:12:21.207587993 +0000 UTC m=+0.137620185 container create 79bd8ab3f0b51dfbdee88e6516c04a67b1631e89e382c41971109419ea98c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:12:21.438 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:21 smithi150 podman[62616]: 2024-04-24 23:12:21.410330031 +0000 UTC m=+0.340362223 container init 79bd8ab3f0b51dfbdee88e6516c04a67b1631e89e382c41971109419ea98c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-24T23:12:21.438 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:21 smithi150 podman[62616]: 2024-04-24 23:12:21.412865802 +0000 UTC m=+0.342898005 container start 79bd8ab3f0b51dfbdee88e6516c04a67b1631e89e382c41971109419ea98c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T23:12:21.438 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mds[62635]: -- 172.21.15.150:0/3836485557 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55570993fa40 con 0x555709912c00 2024-04-24T23:12:21.438 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: starting mds.b at 2024-04-24T23:12:21.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.648+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffad402d370 con 0x7ffad4102980 2024-04-24T23:12:21.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.649+0000 7ffacaffd640 1 -- 172.21.15.112:0/2973059069 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9668 (secure 0 0 0) 0x7ffac40bbed0 con 0x7ffad4102980 2024-04-24T23:12:21.650 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:21.650 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-24T23:12:21.650 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffac00a2c10 msgr2=0x7ffac00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffac00a2c10 0x7ffac00a50d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffad00058c0 tx=0x7ffad0005d50 comp rx=0 tx=0).stop 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 msgr2=0x7ffad41980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 0x7ffad41980d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009a20 tx=0x7ffac4004270 comp rx=0 tx=0).stop 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 shutdown_connections 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffac00a2c10 0x7ffac00a50d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad4103340 0x7ffad419f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4102980 0x7ffad41980d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 --2- 172.21.15.112:0/2973059069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4101790 0x7ffad4197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 >> 172.21.15.112:0/2973059069 conn(0x7ffad40fcd40 msgr2=0x7ffad4105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 shutdown_connections 2024-04-24T23:12:21.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:21.651+0000 7ffadc189640 1 -- 172.21.15.112:0/2973059069 wait complete. 2024-04-24T23:12:21.818 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:21 smithi150 bash[62616]: 79bd8ab3f0b51dfbdee88e6516c04a67b1631e89e382c41971109419ea98c011 2024-04-24T23:12:21.818 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:21 smithi150 systemd[1]: Started Ceph mds.b for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2973059069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:21 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2973059069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:21 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:22.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:22.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2973059069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:22.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:22.427 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:21 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:23.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] up:boot 2024-04-24T23:12:23.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:22 smithi150 ceph-mon[32247]: fsmap 12 up:standby 2024-04-24T23:12:23.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:12:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] up:boot 2024-04-24T23:12:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:22 smithi187 ceph-mon[33292]: fsmap 12 up:standby 2024-04-24T23:12:23.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:12:23.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] up:boot 2024-04-24T23:12:23.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:22 smithi112 ceph-mon[29186]: fsmap 12 up:standby 2024-04-24T23:12:23.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:12:23.664 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 -- 172.21.15.112:0/392902076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134104b30 msgr2=0x7fa134106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 --2- 172.21.15.112:0/392902076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134104b30 0x7fa134106f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009900 tx=0x7fa11c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 -- 172.21.15.112:0/392902076 shutdown_connections 2024-04-24T23:12:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 --2- 172.21.15.112:0/392902076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa134107460 0x7fa1341098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 --2- 172.21.15.112:0/392902076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134104b30 0x7fa134106f20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 --2- 172.21.15.112:0/392902076 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134102200 0x7fa1341045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 -- 172.21.15.112:0/392902076 >> 172.21.15.112:0/392902076 conn(0x7fa1340fbbf0 msgr2=0x7fa1340fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 -- 172.21.15.112:0/392902076 shutdown_connections 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.763+0000 7fa138efc640 1 -- 172.21.15.112:0/392902076 wait complete. 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa138efc640 1 Processor -- start 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa138efc640 1 -- start start 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa138efc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134102200 0x7fa134197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa138efc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134104b30 0x7fa1341980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa138efc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa134107460 0x7fa13419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa138efc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa13410d580 con 0x7fa134102200 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa138efc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa13410d400 con 0x7fa134104b30 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa138efc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa13410d700 con 0x7fa134107460 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.764+0000 7fa13259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134102200 0x7fa134197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa13259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134102200 0x7fa134197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41380/0 (socket says 172.21.15.112:41380) 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa13259c640 1 -- 172.21.15.112:0/463570409 learned_addr learned my addr 172.21.15.112:0/463570409 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa131d9b640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134104b30 0x7fa1341980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa132d9d640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa134107460 0x7fa13419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa13259c640 1 -- 172.21.15.112:0/463570409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa134107460 msgr2=0x7fa13419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa13259c640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa134107460 0x7fa13419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa13259c640 1 -- 172.21.15.112:0/463570409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134104b30 msgr2=0x7fa1341980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa13259c640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134104b30 0x7fa1341980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa13259c640 1 -- 172.21.15.112:0/463570409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13419f790 con 0x7fa134102200 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa131d9b640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134104b30 0x7fa1341980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa13259c640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134102200 0x7fa134197b70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa12400ebc0 tx=0x7fa12400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa12b7fe640 1 -- 172.21.15.112:0/463570409 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124010630 con 0x7fa134102200 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13419fa80 con 0x7fa134102200 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa12b7fe640 1 -- 172.21.15.112:0/463570409 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa124010cd0 con 0x7fa134102200 2024-04-24T23:12:23.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa12b7fe640 1 -- 172.21.15.112:0/463570409 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124018cf0 con 0x7fa134102200 2024-04-24T23:12:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.765+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13419ffa0 con 0x7fa134102200 2024-04-24T23:12:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.766+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1340fd6e0 con 0x7fa134102200 2024-04-24T23:12:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.767+0000 7fa12b7fe640 1 -- 172.21.15.112:0/463570409 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa1240107d0 con 0x7fa134102200 2024-04-24T23:12:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.768+0000 7fa12b7fe640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1200a2b90 0x7fa1200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.768+0000 7fa131d9b640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1200a2b90 0x7fa1200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.768+0000 7fa12b7fe640 1 -- 172.21.15.112:0/463570409 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1240cf6d0 con 0x7fa134102200 2024-04-24T23:12:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.768+0000 7fa131d9b640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1200a2b90 0x7fa1200a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa11c02aa00 tx=0x7fa11c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.771+0000 7fa12b7fe640 1 -- 172.21.15.112:0/463570409 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa12409c480 con 0x7fa134102200 2024-04-24T23:12:23.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.988+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa13402d170 con 0x7fa134102200 2024-04-24T23:12:23.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.989+0000 7fa12b7fe640 1 -- 172.21.15.112:0/463570409 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7fa1240a0780 con 0x7fa134102200 2024-04-24T23:12:23.990 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-24T23:12:23.990 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-04-24T23:12:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1200a2b90 msgr2=0x7fa1200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1200a2b90 0x7fa1200a5050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa11c02aa00 tx=0x7fa11c005a20 comp rx=0 tx=0).stop 2024-04-24T23:12:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134102200 msgr2=0x7fa134197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134102200 0x7fa134197b70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa12400ebc0 tx=0x7fa12400c6a0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 shutdown_connections 2024-04-24T23:12:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1200a2b90 0x7fa1200a5050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa134107460 0x7fa13419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa134104b30 0x7fa1341980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 --2- 172.21.15.112:0/463570409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa134102200 0x7fa134197b70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 >> 172.21.15.112:0/463570409 conn(0x7fa1340fbbf0 msgr2=0x7fa134102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 shutdown_connections 2024-04-24T23:12:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:23.991+0000 7fa138efc640 1 -- 172.21.15.112:0/463570409 wait complete. 2024-04-24T23:12:24.002 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:23 smithi112 ceph-mon[29186]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:24.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.004 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:23 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.006 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 -- 172.21.15.112:0/3447927803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5641074e0 msgr2=0x7fd5641098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 --2- 172.21.15.112:0/3447927803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5641074e0 0x7fd5641098d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009ca0 tx=0x7fd55c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 -- 172.21.15.112:0/3447927803 shutdown_connections 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 --2- 172.21.15.112:0/3447927803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5641074e0 0x7fd5641098d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 --2- 172.21.15.112:0/3447927803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd564104bb0 0x7fd564106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 --2- 172.21.15.112:0/3447927803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564102280 0x7fd564104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 -- 172.21.15.112:0/3447927803 >> 172.21.15.112:0/3447927803 conn(0x7fd5640fbbf0 msgr2=0x7fd5640fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 -- 172.21.15.112:0/3447927803 shutdown_connections 2024-04-24T23:12:24.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.104+0000 7fd56bca5640 1 -- 172.21.15.112:0/3447927803 wait complete. 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56bca5640 1 Processor -- start 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56bca5640 1 -- start start 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56bca5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564102280 0x7fd564197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56bca5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564104bb0 0x7fd564197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56bca5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641074e0 0x7fd56419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56bca5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd56410d580 con 0x7fd564104bb0 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56bca5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd56410d400 con 0x7fd5641074e0 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56bca5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd56410d700 con 0x7fd564102280 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd569240640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564104bb0 0x7fd564197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd569240640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564104bb0 0x7fd564197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41398/0 (socket says 172.21.15.112:41398) 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd569a41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564102280 0x7fd564197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd569240640 1 -- 172.21.15.112:0/4107177467 learned_addr learned my addr 172.21.15.112:0/4107177467 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd56a242640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641074e0 0x7fd56419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.105+0000 7fd569240640 1 -- 172.21.15.112:0/4107177467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564102280 msgr2=0x7fd564197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd569240640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564102280 0x7fd564197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd569240640 1 -- 172.21.15.112:0/4107177467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641074e0 msgr2=0x7fd56419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd569240640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641074e0 0x7fd56419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd569240640 1 -- 172.21.15.112:0/4107177467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56419f6d0 con 0x7fd564104bb0 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd56a242640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641074e0 0x7fd56419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd569240640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564104bb0 0x7fd564197ff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd55400e9c0 tx=0x7fd55400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd55affd640 1 -- 172.21.15.112:0/4107177467 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55400ccc0 con 0x7fd564104bb0 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56419f9c0 con 0x7fd564104bb0 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd55affd640 1 -- 172.21.15.112:0/4107177467 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd554005520 con 0x7fd564104bb0 2024-04-24T23:12:24.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd55affd640 1 -- 172.21.15.112:0/4107177467 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554010690 con 0x7fd564104bb0 2024-04-24T23:12:24.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.106+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56419ff60 con 0x7fd564104bb0 2024-04-24T23:12:24.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.107+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5640fd6e0 con 0x7fd564104bb0 2024-04-24T23:12:24.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.109+0000 7fd55affd640 1 -- 172.21.15.112:0/4107177467 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd554004e40 con 0x7fd564104bb0 2024-04-24T23:12:24.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.109+0000 7fd55affd640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd5500a2b90 0x7fd5500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.110+0000 7fd55affd640 1 -- 172.21.15.112:0/4107177467 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd5540cf810 con 0x7fd564104bb0 2024-04-24T23:12:24.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.110+0000 7fd569a41640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd5500a2b90 0x7fd5500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.110+0000 7fd569a41640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd5500a2b90 0x7fd5500a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd56000abe0 tx=0x7fd56000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.113+0000 7fd55affd640 1 -- 172.21.15.112:0/4107177467 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd55409cc10 con 0x7fd564104bb0 2024-04-24T23:12:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:23 smithi187 ceph-mon[33292]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:23 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.297+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd56402d390 con 0x7fd564104bb0 2024-04-24T23:12:24.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.298+0000 7fd55affd640 1 -- 172.21.15.112:0/4107177467 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18515 (secure 0 0 0) 0x7fd55409c510 con 0x7fd564104bb0 2024-04-24T23:12:24.299 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:24.299 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":82,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:11:38.636461+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:24.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd5500a2b90 msgr2=0x7fd5500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd5500a2b90 0x7fd5500a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd56000abe0 tx=0x7fd56000a2d0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564104bb0 msgr2=0x7fd564197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564104bb0 0x7fd564197ff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd55400e9c0 tx=0x7fd55400edc0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 shutdown_connections 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd5500a2b90 0x7fd5500a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd5641074e0 0x7fd56419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd564104bb0 0x7fd564197ff0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 --2- 172.21.15.112:0/4107177467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564102280 0x7fd564197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 >> 172.21.15.112:0/4107177467 conn(0x7fd5640fbbf0 msgr2=0x7fd564102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 shutdown_connections 2024-04-24T23:12:24.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.300+0000 7fd56bca5640 1 -- 172.21.15.112:0/4107177467 wait complete. 2024-04-24T23:12:24.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:12:24.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:24.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:23 smithi150 ceph-mon[32247]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:23 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/1284077473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4107460 msgr2=0x7f3cf41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/1284077473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4107460 0x7f3cf41098b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0009900 tx=0x7f3cf002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/1284077473 shutdown_connections 2024-04-24T23:12:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/1284077473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4107460 0x7f3cf41098b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/1284077473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf4104b30 0x7f3cf4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/1284077473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf4102200 0x7f3cf41045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/1284077473 >> 172.21.15.112:0/1284077473 conn(0x7f3cf40fbbf0 msgr2=0x7f3cf40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/1284077473 shutdown_connections 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/1284077473 wait complete. 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 Processor -- start 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.415+0000 7f3cfc1f4640 1 -- start start 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfc1f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4102200 0x7f3cf4101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfc1f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf4104b30 0x7f3cf41001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfc1f4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf4107460 0x7f3cf41006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfc1f4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf410d550 con 0x7f3cf4102200 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfc1f4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf410d3d0 con 0x7f3cf4107460 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfc1f4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf410d6d0 con 0x7f3cf4104b30 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4102200 0x7f3cf4101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4102200 0x7f3cf4101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41426/0 (socket says 172.21.15.112:41426) 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 -- 172.21.15.112:0/329629853 learned_addr learned my addr 172.21.15.112:0/329629853 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:24.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf978f640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf4104b30 0x7f3cf41001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfa791640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf4107460 0x7f3cf41006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 -- 172.21.15.112:0/329629853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf4104b30 msgr2=0x7f3cf41001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf4104b30 0x7f3cf41001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 -- 172.21.15.112:0/329629853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf4107460 msgr2=0x7f3cf41006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf4107460 0x7f3cf41006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 -- 172.21.15.112:0/329629853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf4100f80 con 0x7f3cf4102200 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cf9f90640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4102200 0x7f3cf4101b70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00ebc0 tx=0x7f3cec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3ceaffd640 1 -- 172.21.15.112:0/329629853 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec010630 con 0x7f3cf4102200 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf41a1c80 con 0x7f3cf4102200 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3ceaffd640 1 -- 172.21.15.112:0/329629853 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3cec010cd0 con 0x7f3cf4102200 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3ceaffd640 1 -- 172.21.15.112:0/329629853 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec018cf0 con 0x7f3cf4102200 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.416+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf41a21d0 con 0x7f3cf4102200 2024-04-24T23:12:24.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.417+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf40fd6e0 con 0x7f3cf4102200 2024-04-24T23:12:24.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.419+0000 7f3ceaffd640 1 -- 172.21.15.112:0/329629853 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3cec0107d0 con 0x7f3cf4102200 2024-04-24T23:12:24.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.419+0000 7f3ceaffd640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3ce00a2b90 0x7f3ce00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.419+0000 7f3cf978f640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3ce00a2b90 0x7f3ce00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.419+0000 7f3ceaffd640 1 -- 172.21.15.112:0/329629853 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3cec0cf6d0 con 0x7f3cf4102200 2024-04-24T23:12:24.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.420+0000 7f3cf978f640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3ce00a2b90 0x7f3ce00a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cf41018e0 tx=0x7f3ce4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.427+0000 7f3ceaffd640 1 -- 172.21.15.112:0/329629853 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3cec09c480 con 0x7f3cf4102200 2024-04-24T23:12:24.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.620+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3cf402d300 con 0x7f3cf4102200 2024-04-24T23:12:24.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.621+0000 7f3ceaffd640 1 -- 172.21.15.112:0/329629853 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f3cec0a0780 con 0x7f3cf4102200 2024-04-24T23:12:24.622 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:12:24.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3ce00a2b90 msgr2=0x7f3ce00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3ce00a2b90 0x7f3ce00a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3cf41018e0 tx=0x7f3ce4005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:24.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4102200 msgr2=0x7f3cf4101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4102200 0x7f3cf4101b70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00ebc0 tx=0x7f3cec00c6a0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 shutdown_connections 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3ce00a2b90 0x7f3ce00a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cf4107460 0x7f3cf41006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cf4104b30 0x7f3cf41001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 --2- 172.21.15.112:0/329629853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3cf4102200 0x7f3cf4101b70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 >> 172.21.15.112:0/329629853 conn(0x7f3cf40fbbf0 msgr2=0x7f3cf4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 shutdown_connections 2024-04-24T23:12:24.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.623+0000 7f3cfc1f4640 1 -- 172.21.15.112:0/329629853 wait complete. 2024-04-24T23:12:24.635 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-24T23:12:24.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 -- 172.21.15.112:0/1014223923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01018f0 msgr2=0x7f02d0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 --2- 172.21.15.112:0/1014223923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01018f0 0x7f02d0101d70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009900 tx=0x7f02c002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:24.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 -- 172.21.15.112:0/1014223923 shutdown_connections 2024-04-24T23:12:24.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 --2- 172.21.15.112:0/1014223923 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02d01022b0 0x7f02d0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 --2- 172.21.15.112:0/1014223923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01018f0 0x7f02d0101d70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 --2- 172.21.15.112:0/1014223923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02d0100700 0x7f02d0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 -- 172.21.15.112:0/1014223923 >> 172.21.15.112:0/1014223923 conn(0x7f02d00fbc90 msgr2=0x7f02d00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 -- 172.21.15.112:0/1014223923 shutdown_connections 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 -- 172.21.15.112:0/1014223923 wait complete. 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.740+0000 7f02d6238640 1 Processor -- start 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 -- start start 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02d0100700 0x7f02d0104760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02d01018f0 0x7f02d0102da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01022b0 0x7f02d01032e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02d010c440 con 0x7f02d01022b0 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f02d010c2c0 con 0x7f02d01018f0 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f02d010c5c0 con 0x7f02d0100700 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01022b0 0x7f02d01032e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01022b0 0x7f02d01032e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41448/0 (socket says 172.21.15.112:41448) 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 -- 172.21.15.112:0/3446958064 learned_addr learned my addr 172.21.15.112:0/3446958064 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cf7fe640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02d0100700 0x7f02d0104760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02ceffd640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02d01018f0 0x7f02d0102da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 -- 172.21.15.112:0/3446958064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02d0100700 msgr2=0x7f02d0104760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02d0100700 0x7f02d0104760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 -- 172.21.15.112:0/3446958064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02d01018f0 msgr2=0x7f02d0102da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02d01018f0 0x7f02d0102da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 -- 172.21.15.112:0/3446958064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d0103b70 con 0x7f02d01022b0 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02cffff640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01022b0 0x7f02d01032e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f02c800eaa0 tx=0x7f02c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02ccff9640 1 -- 172.21.15.112:0/3446958064 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c800cdb0 con 0x7f02d01022b0 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d01a1c80 con 0x7f02d01022b0 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02ccff9640 1 -- 172.21.15.112:0/3446958064 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f02c8004b20 con 0x7f02d01022b0 2024-04-24T23:12:24.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02ccff9640 1 -- 172.21.15.112:0/3446958064 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8011760 con 0x7f02d01022b0 2024-04-24T23:12:24.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.741+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d01a21a0 con 0x7f02d01022b0 2024-04-24T23:12:24.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.743+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d00689b0 con 0x7f02d01022b0 2024-04-24T23:12:24.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.744+0000 7f02ccff9640 1 -- 172.21.15.112:0/3446958064 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f02c8004cd0 con 0x7f02d01022b0 2024-04-24T23:12:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.744+0000 7f02ccff9640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f02b80a2b90 0x7f02b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.744+0000 7f02cf7fe640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f02b80a2b90 0x7f02b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.745+0000 7f02ccff9640 1 -- 172.21.15.112:0/3446958064 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f02c80ced80 con 0x7f02d01022b0 2024-04-24T23:12:24.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.745+0000 7f02cf7fe640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f02b80a2b90 0x7f02b80a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f02c4005fd0 tx=0x7f02c400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:24.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.749+0000 7f02ccff9640 1 -- 172.21.15.112:0/3446958064 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f02c8016030 con 0x7f02d01022b0 2024-04-24T23:12:24.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.930+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f02d01a25e0 con 0x7f02d01022b0 2024-04-24T23:12:24.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.931+0000 7f02ccff9640 1 -- 172.21.15.112:0/3446958064 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f02c809bb30 con 0x7f02d01022b0 2024-04-24T23:12:24.933 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:24.933 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-04-24T23:12:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:12:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-04-24T23:12:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-04-24T23:12:24.934 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:12:24.934 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-04-24T23:12:24.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f02b80a2b90 msgr2=0x7f02b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f02b80a2b90 0x7f02b80a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f02c4005fd0 tx=0x7f02c400b390 comp rx=0 tx=0).stop 2024-04-24T23:12:24.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01022b0 msgr2=0x7f02d01032e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:24.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01022b0 0x7f02d01032e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f02c800eaa0 tx=0x7f02c800ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:24.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 shutdown_connections 2024-04-24T23:12:24.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f02b80a2b90 0x7f02b80a5050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02d01022b0 0x7f02d01032e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02d01018f0 0x7f02d0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 --2- 172.21.15.112:0/3446958064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02d0100700 0x7f02d0104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 >> 172.21.15.112:0/3446958064 conn(0x7f02d00fbc90 msgr2=0x7f02d0069dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 shutdown_connections 2024-04-24T23:12:24.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:24.933+0000 7f02d6238640 1 -- 172.21.15.112:0/3446958064 wait complete. 2024-04-24T23:12:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:24 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/463570409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:24 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:24 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4107177467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:24.942 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:24 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/329629853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:24.947 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-24T23:12:24.947 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 -- 172.21.15.112:0/591369961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30681006a0 msgr2=0x7f3068100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 --2- 172.21.15.112:0/591369961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30681006a0 0x7f3068100ac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3058009900 tx=0x7f305802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 -- 172.21.15.112:0/591369961 shutdown_connections 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 --2- 172.21.15.112:0/591369961 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3068102250 0x7f3068108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 --2- 172.21.15.112:0/591369961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3068101890 0x7f3068101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 --2- 172.21.15.112:0/591369961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30681006a0 0x7f3068100ac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 -- 172.21.15.112:0/591369961 >> 172.21.15.112:0/591369961 conn(0x7f30680fbc30 msgr2=0x7f30680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 -- 172.21.15.112:0/591369961 shutdown_connections 2024-04-24T23:12:25.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.050+0000 7f306e930640 1 -- 172.21.15.112:0/591369961 wait complete. 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306e930640 1 Processor -- start 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306e930640 1 -- start start 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306e930640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30681006a0 0x7f3068197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306e930640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3068101890 0x7f30681981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306e930640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3068102250 0x7f306819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306e930640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f306810c450 con 0x7f3068102250 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306e930640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f306810c2d0 con 0x7f3068101890 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306e930640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f306810c5d0 con 0x7f30681006a0 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306cecd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3068102250 0x7f306819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306cecd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3068102250 0x7f306819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41470/0 (socket says 172.21.15.112:41470) 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f306cecd640 1 -- 172.21.15.112:0/776993984 learned_addr learned my addr 172.21.15.112:0/776993984 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:25.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.051+0000 7f3067fff640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30681006a0 0x7f3068197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f30677fe640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3068101890 0x7f30681981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f306cecd640 1 -- 172.21.15.112:0/776993984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30681006a0 msgr2=0x7f3068197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f306cecd640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30681006a0 0x7f3068197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f306cecd640 1 -- 172.21.15.112:0/776993984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3068101890 msgr2=0x7f30681981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f306cecd640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3068101890 0x7f30681981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f306cecd640 1 -- 172.21.15.112:0/776993984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306819f830 con 0x7f3068102250 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f306cecd640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3068102250 0x7f306819f190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f306000eb10 tx=0x7f306000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f30657fa640 1 -- 172.21.15.112:0/776993984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060004b20 con 0x7f3068102250 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f30657fa640 1 -- 172.21.15.112:0/776993984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3060004cc0 con 0x7f3068102250 2024-04-24T23:12:25.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306819fb20 con 0x7f3068102250 2024-04-24T23:12:25.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f30657fa640 1 -- 172.21.15.112:0/776993984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30600107f0 con 0x7f3068102250 2024-04-24T23:12:25.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.052+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30681a0070 con 0x7f3068102250 2024-04-24T23:12:25.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.054+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30681a02a0 con 0x7f3068102250 2024-04-24T23:12:25.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.055+0000 7f30657fa640 1 -- 172.21.15.112:0/776993984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3060005020 con 0x7f3068102250 2024-04-24T23:12:25.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.056+0000 7f30657fa640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f30500a2c10 0x7f30500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:25.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.056+0000 7f30657fa640 1 -- 172.21.15.112:0/776993984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f30600cf1a0 con 0x7f3068102250 2024-04-24T23:12:25.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.056+0000 7f3067fff640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f30500a2c10 0x7f30500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:25.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.056+0000 7f3067fff640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f30500a2c10 0x7f30500a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f30681990f0 tx=0x7f3058031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:25.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.060+0000 7f30657fa640 1 -- 172.21.15.112:0/776993984 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f306009c5a0 con 0x7f3068102250 2024-04-24T23:12:25.145 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:24 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:25.145 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:24 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:25.145 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:24 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:25.145 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:24 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/463570409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.145 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:24 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.145 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:24 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4107177467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.145 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:24 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/329629853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:24 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/463570409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:24 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:24 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4107177467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:24 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/329629853' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:25.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.241+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f30681a05f0 con 0x7f3068102250 2024-04-24T23:12:25.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.973+0000 7f30657fa640 1 -- 172.21.15.112:0/776993984 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f306009bea0 con 0x7f3068102250 2024-04-24T23:12:25.975 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f30500a2c10 msgr2=0x7f30500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f30500a2c10 0x7f30500a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f30681990f0 tx=0x7f3058031640 comp rx=0 tx=0).stop 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3068102250 msgr2=0x7f306819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3068102250 0x7f306819f190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f306000eb10 tx=0x7f306000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 shutdown_connections 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f30500a2c10 0x7f30500a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3068102250 0x7f306819f190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3068101890 0x7f30681981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 --2- 172.21.15.112:0/776993984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30681006a0 0x7f3068197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 >> 172.21.15.112:0/776993984 conn(0x7f30680fbc30 msgr2=0x7f3068069ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 shutdown_connections 2024-04-24T23:12:25.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:25.975+0000 7f306e930640 1 -- 172.21.15.112:0/776993984 wait complete. 2024-04-24T23:12:25.991 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-24T23:12:26.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 -- 172.21.15.112:0/3756418138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 msgr2=0x7fccb0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 --2- 172.21.15.112:0/3756418138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 0x7fccb0106f20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009900 tx=0x7fcc9802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:26.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 -- 172.21.15.112:0/3756418138 shutdown_connections 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 --2- 172.21.15.112:0/3756418138 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0107460 0x7fccb01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 --2- 172.21.15.112:0/3756418138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 0x7fccb0106f20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 --2- 172.21.15.112:0/3756418138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb0102200 0x7fccb01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 -- 172.21.15.112:0/3756418138 >> 172.21.15.112:0/3756418138 conn(0x7fccb00fbbf0 msgr2=0x7fccb00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 -- 172.21.15.112:0/3756418138 shutdown_connections 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.094+0000 7fccb5932640 1 -- 172.21.15.112:0/3756418138 wait complete. 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb5932640 1 Processor -- start 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb5932640 1 -- start start 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb5932640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0102200 0x7fccb0197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb5932640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 0x7fccb0198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb5932640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb0107460 0x7fccb019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb5932640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fccb010d590 con 0x7fccb0104b30 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb5932640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fccb010d410 con 0x7fccb0102200 2024-04-24T23:12:26.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb5932640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fccb010d710 con 0x7fccb0107460 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 0x7fccb0198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 0x7fccb0198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41488/0 (socket says 172.21.15.112:41488) 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 -- 172.21.15.112:0/4046465393 learned_addr learned my addr 172.21.15.112:0/4046465393 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccaf7fe640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb0107460 0x7fccb019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 -- 172.21.15.112:0/4046465393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb0107460 msgr2=0x7fccb019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb0107460 0x7fccb019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccaeffd640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0102200 0x7fccb0197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 -- 172.21.15.112:0/4046465393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0102200 msgr2=0x7fccb0197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0102200 0x7fccb0197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 -- 172.21.15.112:0/4046465393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb019f8d0 con 0x7fccb0104b30 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccaeffd640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0102200 0x7fccb0197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fcca7fff640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 0x7fccb0198220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009230 tx=0x7fcc98004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.095+0000 7fccb4930640 1 -- 172.21.15.112:0/4046465393 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9803f030 con 0x7fccb0104b30 2024-04-24T23:12:26.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.096+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb019fb60 con 0x7fccb0104b30 2024-04-24T23:12:26.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.096+0000 7fccb4930640 1 -- 172.21.15.112:0/4046465393 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc98003d00 con 0x7fccb0104b30 2024-04-24T23:12:26.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.096+0000 7fccb4930640 1 -- 172.21.15.112:0/4046465393 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc98035070 con 0x7fccb0104b30 2024-04-24T23:12:26.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.096+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb01a00a0 con 0x7fccb0104b30 2024-04-24T23:12:26.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.096+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb00fd6e0 con 0x7fccb0104b30 2024-04-24T23:12:26.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.098+0000 7fccb4930640 1 -- 172.21.15.112:0/4046465393 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcc98042030 con 0x7fccb0104b30 2024-04-24T23:12:26.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.098+0000 7fccb4930640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc9c0a2b90 0x7fcc9c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:26.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.098+0000 7fccaeffd640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc9c0a2b90 0x7fcc9c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:26.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.098+0000 7fccb4930640 1 -- 172.21.15.112:0/4046465393 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fcc980ef8a0 con 0x7fccb0104b30 2024-04-24T23:12:26.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.099+0000 7fccaeffd640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc9c0a2b90 0x7fcc9c0a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcca0006fd0 tx=0x7fcca0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:26.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.102+0000 7fccb4930640 1 -- 172.21.15.112:0/4046465393 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc980bcb30 con 0x7fccb0104b30 2024-04-24T23:12:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3446958064' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:12:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/776993984' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:26.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:25 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3446958064' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/776993984' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:25 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.279+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fccb0062340 con 0x7fccb0104b30 2024-04-24T23:12:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3446958064' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-24T23:12:26.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/776993984' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-24T23:12:26.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:26.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:12:26.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:25 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:12:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.980+0000 7fccb4930640 1 -- 172.21.15.112:0/4046465393 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7fcc980bc430 con 0x7fccb0104b30 2024-04-24T23:12:26.982 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-04-24T23:12:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc9c0a2b90 msgr2=0x7fcc9c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc9c0a2b90 0x7fcc9c0a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcca0006fd0 tx=0x7fcca0008040 comp rx=0 tx=0).stop 2024-04-24T23:12:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 msgr2=0x7fccb0198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 0x7fccb0198220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc98009230 tx=0x7fcc98004430 comp rx=0 tx=0).stop 2024-04-24T23:12:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 shutdown_connections 2024-04-24T23:12:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc9c0a2b90 0x7fcc9c0a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fccb0107460 0x7fccb019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccb0104b30 0x7fccb0198220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 --2- 172.21.15.112:0/4046465393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0102200 0x7fccb0197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 >> 172.21.15.112:0/4046465393 conn(0x7fccb00fbbf0 msgr2=0x7fccb0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 shutdown_connections 2024-04-24T23:12:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:26.982+0000 7fccb5932640 1 -- 172.21.15.112:0/4046465393 wait complete. 2024-04-24T23:12:27.004 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-24T23:12:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 -- 172.21.15.112:0/4228546749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b8102990 msgr2=0x7f27b8102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 --2- 172.21.15.112:0/4228546749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b8102990 0x7f27b8102e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009900 tx=0x7f27a802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 -- 172.21.15.112:0/4228546749 shutdown_connections 2024-04-24T23:12:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 --2- 172.21.15.112:0/4228546749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27b8103350 0x7f27b8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 --2- 172.21.15.112:0/4228546749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b8102990 0x7f27b8102e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 --2- 172.21.15.112:0/4228546749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b81017a0 0x7f27b8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 -- 172.21.15.112:0/4228546749 >> 172.21.15.112:0/4228546749 conn(0x7f27b80fcd10 msgr2=0x7f27b80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 -- 172.21.15.112:0/4228546749 shutdown_connections 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 -- 172.21.15.112:0/4228546749 wait complete. 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 Processor -- start 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.106+0000 7f27c07b7640 1 -- start start 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27c07b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b81017a0 0x7f27b8197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27c07b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27b8102990 0x7f27b81980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27c07b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8103350 0x7f27b819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27c07b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810d550 con 0x7f27b81017a0 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27c07b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810d3d0 con 0x7f27b8103350 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27c07b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27b810d6d0 con 0x7f27b8102990 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b81017a0 0x7f27b8197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b81017a0 0x7f27b8197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41512/0 (socket says 172.21.15.112:41512) 2024-04-24T23:12:27.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 -- 172.21.15.112:0/774866788 learned_addr learned my addr 172.21.15.112:0/774866788 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27bed54640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8103350 0x7f27b819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27bdd52640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27b8102990 0x7f27b81980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 -- 172.21.15.112:0/774866788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27b8102990 msgr2=0x7f27b81980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27b8102990 0x7f27b81980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 -- 172.21.15.112:0/774866788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8103350 msgr2=0x7f27b819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8103350 0x7f27b819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 -- 172.21.15.112:0/774866788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b819f780 con 0x7f27b81017a0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27bed54640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8103350 0x7f27b819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27be553640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b81017a0 0x7f27b8197b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f27b400d770 tx=0x7f27b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27af7fe640 1 -- 172.21.15.112:0/774866788 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4017070 con 0x7f27b81017a0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27af7fe640 1 -- 172.21.15.112:0/774866788 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f27b4009c30 con 0x7f27b81017a0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27af7fe640 1 -- 172.21.15.112:0/774866788 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4003e40 con 0x7f27b81017a0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b819fa70 con 0x7f27b81017a0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.107+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b819ffc0 con 0x7f27b81017a0 2024-04-24T23:12:27.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.108+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b8108a60 con 0x7f27b81017a0 2024-04-24T23:12:27.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.110+0000 7f27af7fe640 1 -- 172.21.15.112:0/774866788 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f27b4004000 con 0x7f27b81017a0 2024-04-24T23:12:27.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.110+0000 7f27af7fe640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f27a40a2b90 0x7f27a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:27.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.110+0000 7f27bdd52640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f27a40a2b90 0x7f27a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:27.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.110+0000 7f27af7fe640 1 -- 172.21.15.112:0/774866788 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f27b40cecd0 con 0x7f27b81017a0 2024-04-24T23:12:27.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.111+0000 7f27bdd52640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f27a40a2b90 0x7f27a40a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27a80316a0 tx=0x7f27a8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.114+0000 7f27af7fe640 1 -- 172.21.15.112:0/774866788 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f27b4009de0 con 0x7f27b81017a0 2024-04-24T23:12:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:26 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/776993984' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:12:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:26 smithi187 ceph-mon[33292]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:12:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:26 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4046465393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:12:27.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:26 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/776993984' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:12:27.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:26 smithi112 ceph-mon[29186]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:12:27.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:26 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4046465393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:12:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:26 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/776993984' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-24T23:12:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:26 smithi150 ceph-mon[32247]: osdmap e83: 12 total, 12 up, 12 in 2024-04-24T23:12:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:26 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4046465393' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-24T23:12:27.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.328+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f27b802d170 con 0x7f27b81017a0 2024-04-24T23:12:27.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.982+0000 7f27af7fe640 1 -- 172.21.15.112:0/774866788 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f27b409bde0 con 0x7f27b81017a0 2024-04-24T23:12:27.984 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-04-24T23:12:27.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.984+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f27a40a2b90 msgr2=0x7f27a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.984+0000 7f27c07b7640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f27a40a2b90 0x7f27a40a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27a80316a0 tx=0x7f27a8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b81017a0 msgr2=0x7f27b8197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:27.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b81017a0 0x7f27b8197b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f27b400d770 tx=0x7f27b400dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:27.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 shutdown_connections 2024-04-24T23:12:27.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f27a40a2b90 0x7f27a40a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f27b8103350 0x7f27b819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27b8102990 0x7f27b81980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 --2- 172.21.15.112:0/774866788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27b81017a0 0x7f27b8197b60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:27.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 >> 172.21.15.112:0/774866788 conn(0x7f27b80fcd10 msgr2=0x7f27b8105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:27.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 shutdown_connections 2024-04-24T23:12:27.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:27.985+0000 7f27c07b7640 1 -- 172.21.15.112:0/774866788 wait complete. 2024-04-24T23:12:28.007 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-24T23:12:28.008 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 -- 172.21.15.112:0/4096141474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c107460 msgr2=0x7fbc3c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 --2- 172.21.15.112:0/4096141474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c107460 0x7fbc3c1098b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009ca0 tx=0x7fbc3402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 -- 172.21.15.112:0/4096141474 shutdown_connections 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 --2- 172.21.15.112:0/4096141474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c107460 0x7fbc3c1098b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 --2- 172.21.15.112:0/4096141474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc3c104b30 0x7fbc3c106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 --2- 172.21.15.112:0/4096141474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc3c102200 0x7fbc3c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 -- 172.21.15.112:0/4096141474 >> 172.21.15.112:0/4096141474 conn(0x7fbc3c0fbc30 msgr2=0x7fbc3c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 -- 172.21.15.112:0/4096141474 shutdown_connections 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.114+0000 7fbc420c4640 1 -- 172.21.15.112:0/4096141474 wait complete. 2024-04-24T23:12:28.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc420c4640 1 Processor -- start 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc420c4640 1 -- start start 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc420c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc3c102200 0x7fbc3c197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc420c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c104b30 0x7fbc3c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc420c4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc3c107460 0x7fbc3c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc420c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10d5d0 con 0x7fbc3c104b30 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc420c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10d450 con 0x7fbc3c102200 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc420c4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c10d750 con 0x7fbc3c107460 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc3affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c104b30 0x7fbc3c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc3affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c104b30 0x7fbc3c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41544/0 (socket says 172.21.15.112:41544) 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.115+0000 7fbc3affd640 1 -- 172.21.15.112:0/2043099533 learned_addr learned my addr 172.21.15.112:0/2043099533 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3bfff640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc3c107460 0x7fbc3c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3b7fe640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc3c102200 0x7fbc3c197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3affd640 1 -- 172.21.15.112:0/2043099533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc3c107460 msgr2=0x7fbc3c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3affd640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc3c107460 0x7fbc3c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3affd640 1 -- 172.21.15.112:0/2043099533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc3c102200 msgr2=0x7fbc3c197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3affd640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc3c102200 0x7fbc3c197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3affd640 1 -- 172.21.15.112:0/2043099533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c19f7f0 con 0x7fbc3c104b30 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3bfff640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc3c107460 0x7fbc3c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc3affd640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c104b30 0x7fbc3c198180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000d740 tx=0x7fbc3000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc38ff9640 1 -- 172.21.15.112:0/2043099533 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30017070 con 0x7fbc3c104b30 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc38ff9640 1 -- 172.21.15.112:0/2043099533 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbc3000de50 con 0x7fbc3c104b30 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c19fae0 con 0x7fbc3c104b30 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc38ff9640 1 -- 172.21.15.112:0/2043099533 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30010460 con 0x7fbc3c104b30 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.116+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c1a0060 con 0x7fbc3c104b30 2024-04-24T23:12:28.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.117+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c0fd720 con 0x7fbc3c104b30 2024-04-24T23:12:28.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.118+0000 7fbc38ff9640 1 -- 172.21.15.112:0/2043099533 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbc30004f10 con 0x7fbc3c104b30 2024-04-24T23:12:28.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.118+0000 7fbc38ff9640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbc240a2b90 0x7fbc240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:28.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.119+0000 7fbc3b7fe640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbc240a2b90 0x7fbc240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:28.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.119+0000 7fbc38ff9640 1 -- 172.21.15.112:0/2043099533 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbc300ce990 con 0x7fbc3c104b30 2024-04-24T23:12:28.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.119+0000 7fbc3b7fe640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbc240a2b90 0x7fbc240a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009890 tx=0x7fbc2c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:28.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.123+0000 7fbc38ff9640 1 -- 172.21.15.112:0/2043099533 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc3009ba40 con 0x7fbc3c104b30 2024-04-24T23:12:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:27 smithi112 ceph-mon[29186]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:27 smithi112 ceph-mon[29186]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:12:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:27 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4046465393' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:12:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:27 smithi112 ceph-mon[29186]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:12:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:27 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/774866788' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:12:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:27 smithi112 ceph-mon[29186]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T23:12:28.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:28.309+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fbc3c02d3c0 con 0x7fbc3c104b30 2024-04-24T23:12:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:27 smithi150 ceph-mon[32247]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:27 smithi150 ceph-mon[32247]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:12:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:27 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4046465393' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:12:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:27 smithi150 ceph-mon[32247]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:12:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:27 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/774866788' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:12:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:27 smithi150 ceph-mon[32247]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T23:12:28.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:27 smithi187 ceph-mon[33292]: pgmap v283: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:12:28.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:27 smithi187 ceph-mon[33292]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:12:28.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:27 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4046465393' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-24T23:12:28.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:27 smithi187 ceph-mon[33292]: osdmap e84: 12 total, 12 up, 12 in 2024-04-24T23:12:28.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:27 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/774866788' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-24T23:12:28.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:27 smithi187 ceph-mon[33292]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-24T23:12:29.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.000+0000 7fbc38ff9640 1 -- 172.21.15.112:0/2043099533 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fbc3009b340 con 0x7fbc3c104b30 2024-04-24T23:12:29.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbc240a2b90 msgr2=0x7fbc240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbc240a2b90 0x7fbc240a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009890 tx=0x7fbc2c008040 comp rx=0 tx=0).stop 2024-04-24T23:12:29.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c104b30 msgr2=0x7fbc3c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c104b30 0x7fbc3c198180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000d740 tx=0x7fbc3000dc00 comp rx=0 tx=0).stop 2024-04-24T23:12:29.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 shutdown_connections 2024-04-24T23:12:29.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbc240a2b90 0x7fbc240a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc3c107460 0x7fbc3c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc3c104b30 0x7fbc3c198180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 --2- 172.21.15.112:0/2043099533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbc3c102200 0x7fbc3c197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 >> 172.21.15.112:0/2043099533 conn(0x7fbc3c0fbc30 msgr2=0x7fbc3c102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 shutdown_connections 2024-04-24T23:12:29.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.003+0000 7fbc420c4640 1 -- 172.21.15.112:0/2043099533 wait complete. 2024-04-24T23:12:29.019 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-24T23:12:29.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- 172.21.15.112:0/2686078815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 msgr2=0x7f7030100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 --2- 172.21.15.112:0/2686078815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 0x7f7030100b20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f702c009900 tx=0x7f702c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:29.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- 172.21.15.112:0/2686078815 shutdown_connections 2024-04-24T23:12:29.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 --2- 172.21.15.112:0/2686078815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70301022b0 0x7f7030108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 --2- 172.21.15.112:0/2686078815 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70301018f0 0x7f7030101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 --2- 172.21.15.112:0/2686078815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 0x7f7030100b20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- 172.21.15.112:0/2686078815 >> 172.21.15.112:0/2686078815 conn(0x7f70300fbc90 msgr2=0x7f70300fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- 172.21.15.112:0/2686078815 shutdown_connections 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- 172.21.15.112:0/2686078815 wait complete. 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 Processor -- start 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- start start 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 0x7f7030197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70301018f0 0x7f70301980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70301022b0 0x7f703019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f703010d5a0 con 0x7f7030100700 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f703010d420 con 0x7f70301018f0 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7036da2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f703010d720 con 0x7f70301022b0 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7034b3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 0x7f7030197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7034b3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 0x7f7030197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41552/0 (socket says 172.21.15.112:41552) 2024-04-24T23:12:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.129+0000 7f7034b3e640 1 -- 172.21.15.112:0/1446048263 learned_addr learned my addr 172.21.15.112:0/1446048263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7027fff640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70301018f0 0x7f70301980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f703533f640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70301022b0 0x7f703019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7034b3e640 1 -- 172.21.15.112:0/1446048263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70301022b0 msgr2=0x7f703019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7034b3e640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70301022b0 0x7f703019f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7034b3e640 1 -- 172.21.15.112:0/1446048263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70301018f0 msgr2=0x7f70301980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7034b3e640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70301018f0 0x7f70301980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7034b3e640 1 -- 172.21.15.112:0/1446048263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703019f780 con 0x7f7030100700 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7034b3e640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 0x7f7030197b60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f702c009230 tx=0x7f702c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7025ffb640 1 -- 172.21.15.112:0/1446048263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c03f030 con 0x7f7030100700 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7025ffb640 1 -- 172.21.15.112:0/1446048263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f702c003d00 con 0x7f7030100700 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703019fa10 con 0x7f7030100700 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7025ffb640 1 -- 172.21.15.112:0/1446048263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c035070 con 0x7f7030100700 2024-04-24T23:12:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.130+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703019ffd0 con 0x7f7030100700 2024-04-24T23:12:29.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.131+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70301079c0 con 0x7f7030100700 2024-04-24T23:12:29.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.132+0000 7f7025ffb640 1 -- 172.21.15.112:0/1446048263 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f702c042030 con 0x7f7030100700 2024-04-24T23:12:29.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.133+0000 7f7025ffb640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70180a2c10 0x7f70180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:29.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.133+0000 7f7027fff640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70180a2c10 0x7f70180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:29.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.133+0000 7f7025ffb640 1 -- 172.21.15.112:0/1446048263 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f702c0efc10 con 0x7f7030100700 2024-04-24T23:12:29.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.133+0000 7f7027fff640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70180a2c10 0x7f70180a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7020005e10 tx=0x7f7020005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.136+0000 7f7025ffb640 1 -- 172.21.15.112:0/1446048263 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f702c0bcb40 con 0x7f7030100700 2024-04-24T23:12:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:29 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/774866788' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:12:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:29 smithi112 ceph-mon[29186]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:12:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:29 smithi112 ceph-mon[29186]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T23:12:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:29 smithi112 ceph-mon[29186]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T23:12:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:29 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2043099533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:12:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:29 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] up:active 2024-04-24T23:12:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:29 smithi112 ceph-mon[29186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:29.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:29.313+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f703002d170 con 0x7f7030100700 2024-04-24T23:12:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:29 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/774866788' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:12:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:29 smithi150 ceph-mon[32247]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:12:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:29 smithi150 ceph-mon[32247]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T23:12:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:29 smithi150 ceph-mon[32247]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T23:12:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:29 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2043099533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:12:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:29 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] up:active 2024-04-24T23:12:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:29 smithi150 ceph-mon[32247]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:29.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:29 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/774866788' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-24T23:12:29.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:29 smithi187 ceph-mon[33292]: osdmap e85: 12 total, 12 up, 12 in 2024-04-24T23:12:29.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:29 smithi187 ceph-mon[33292]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-24T23:12:29.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:29 smithi187 ceph-mon[33292]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-24T23:12:29.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:29 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2043099533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-24T23:12:29.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:29 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] up:active 2024-04-24T23:12:29.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:29 smithi187 ceph-mon[33292]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:30 smithi112 ceph-mon[29186]: pgmap v286: 97 pgs: 8 creating+peering, 27 active+clean, 62 unknown; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-24T23:12:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:30 smithi112 ceph-mon[29186]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:12:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:30 smithi112 ceph-mon[29186]: Cluster is now healthy 2024-04-24T23:12:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:30 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2043099533' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:12:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:30 smithi112 ceph-mon[29186]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:12:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:30 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1446048263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:12:30.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:30 smithi150 ceph-mon[32247]: pgmap v286: 97 pgs: 8 creating+peering, 27 active+clean, 62 unknown; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-24T23:12:30.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:30 smithi150 ceph-mon[32247]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:12:30.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:30 smithi150 ceph-mon[32247]: Cluster is now healthy 2024-04-24T23:12:30.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:30 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2043099533' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:12:30.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:30 smithi150 ceph-mon[32247]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:12:30.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:30 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1446048263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:12:30.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:30 smithi187 ceph-mon[33292]: pgmap v286: 97 pgs: 8 creating+peering, 27 active+clean, 62 unknown; 578 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-24T23:12:30.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:30 smithi187 ceph-mon[33292]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:12:30.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:30 smithi187 ceph-mon[33292]: Cluster is now healthy 2024-04-24T23:12:30.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:30 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2043099533' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-24T23:12:30.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:30 smithi187 ceph-mon[33292]: osdmap e86: 12 total, 12 up, 12 in 2024-04-24T23:12:30.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:30 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1446048263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-24T23:12:31.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.004+0000 7f7025ffb640 1 -- 172.21.15.112:0/1446048263 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f702c0bc440 con 0x7f7030100700 2024-04-24T23:12:31.006 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data_ec' created 2024-04-24T23:12:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70180a2c10 msgr2=0x7f70180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70180a2c10 0x7f70180a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7020005e10 tx=0x7f7020005da0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 msgr2=0x7f7030197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 0x7f7030197b60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f702c009230 tx=0x7f702c004430 comp rx=0 tx=0).stop 2024-04-24T23:12:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 shutdown_connections 2024-04-24T23:12:31.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70180a2c10 0x7f70180a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70301022b0 0x7f703019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70301018f0 0x7f70301980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 --2- 172.21.15.112:0/1446048263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7030100700 0x7f7030197b60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 >> 172.21.15.112:0/1446048263 conn(0x7f70300fbc90 msgr2=0x7f7030104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:31.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 shutdown_connections 2024-04-24T23:12:31.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.008+0000 7f7036da2640 1 -- 172.21.15.112:0/1446048263 wait complete. 2024-04-24T23:12:31.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-24T23:12:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 -- 172.21.15.112:0/3949208526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 msgr2=0x7fab48101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 --2- 172.21.15.112:0/3949208526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 0x7fab48101bb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fab38009900 tx=0x7fab3802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 -- 172.21.15.112:0/3949208526 shutdown_connections 2024-04-24T23:12:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 --2- 172.21.15.112:0/3949208526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48103340 0x7fab48109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 --2- 172.21.15.112:0/3949208526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab48102980 0x7fab48102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 --2- 172.21.15.112:0/3949208526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 0x7fab48101bb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 -- 172.21.15.112:0/3949208526 >> 172.21.15.112:0/3949208526 conn(0x7fab480fcd20 msgr2=0x7fab480ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 -- 172.21.15.112:0/3949208526 shutdown_connections 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.128+0000 7fab4da5d640 1 -- 172.21.15.112:0/3949208526 wait complete. 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab4da5d640 1 Processor -- start 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab4da5d640 1 -- start start 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab4da5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 0x7fab48197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab4da5d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab48102980 0x7fab48198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab4da5d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48103340 0x7fab4819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab4da5d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810d540 con 0x7fab48101790 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab4da5d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810d3c0 con 0x7fab48102980 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab4da5d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab4810d6c0 con 0x7fab48103340 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 0x7fab48197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab46ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 0x7fab48197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46514/0 (socket says 172.21.15.112:46514) 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab46ffd640 1 -- 172.21.15.112:0/937991424 learned_addr learned my addr 172.21.15.112:0/937991424 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:31.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab467fc640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab48102980 0x7fab48198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab46ffd640 1 -- 172.21.15.112:0/937991424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48103340 msgr2=0x7fab4819f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab46ffd640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48103340 0x7fab4819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab46ffd640 1 -- 172.21.15.112:0/937991424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab48102980 msgr2=0x7fab48198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab46ffd640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab48102980 0x7fab48198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.129+0000 7fab46ffd640 1 -- 172.21.15.112:0/937991424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4819f830 con 0x7fab48101790 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.130+0000 7fab46ffd640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 0x7fab48197c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab38009230 tx=0x7fab380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.130+0000 7fab4ca5b640 1 -- 172.21.15.112:0/937991424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3803f030 con 0x7fab48101790 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.130+0000 7fab4ca5b640 1 -- 172.21.15.112:0/937991424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fab38003d00 con 0x7fab48101790 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.130+0000 7fab4ca5b640 1 -- 172.21.15.112:0/937991424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38035070 con 0x7fab48101790 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.130+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4819fac0 con 0x7fab48101790 2024-04-24T23:12:31.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.130+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4819ffb0 con 0x7fab48101790 2024-04-24T23:12:31.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.131+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab48108ab0 con 0x7fab48101790 2024-04-24T23:12:31.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.132+0000 7fab4ca5b640 1 -- 172.21.15.112:0/937991424 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fab38042030 con 0x7fab48101790 2024-04-24T23:12:31.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.132+0000 7fab4ca5b640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fab300a2b90 0x7fab300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:31.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.132+0000 7fab467fc640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fab300a2b90 0x7fab300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:31.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.133+0000 7fab4ca5b640 1 -- 172.21.15.112:0/937991424 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7fab380efd90 con 0x7fab48101790 2024-04-24T23:12:31.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.133+0000 7fab467fc640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fab300a2b90 0x7fab300a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009b60 tx=0x7fab3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:31.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.136+0000 7fab4ca5b640 1 -- 172.21.15.112:0/937991424 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab380bcad0 con 0x7fab48101790 2024-04-24T23:12:31.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:31 smithi112 ceph-mon[29186]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:12:31.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:31 smithi112 ceph-mon[29186]: pgmap v289: 97 pgs: 8 creating+peering, 79 active+clean, 10 unknown; 578 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-24T23:12:31.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:31 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1446048263' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:12:31.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:31 smithi112 ceph-mon[29186]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:12:31.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:31.312+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fab480623e0 con 0x7fab48101790 2024-04-24T23:12:31.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:31 smithi150 ceph-mon[32247]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:12:31.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:31 smithi150 ceph-mon[32247]: pgmap v289: 97 pgs: 8 creating+peering, 79 active+clean, 10 unknown; 578 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-24T23:12:31.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:31 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1446048263' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:12:31.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:31 smithi150 ceph-mon[32247]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:12:31.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:31 smithi187 ceph-mon[33292]: osdmap e87: 12 total, 12 up, 12 in 2024-04-24T23:12:31.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:31 smithi187 ceph-mon[33292]: pgmap v289: 97 pgs: 8 creating+peering, 79 active+clean, 10 unknown; 578 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 4 op/s 2024-04-24T23:12:31.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:31 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1446048263' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-24T23:12:31.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:31 smithi187 ceph-mon[33292]: osdmap e88: 12 total, 12 up, 12 in 2024-04-24T23:12:32.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.006+0000 7fab4ca5b640 1 -- 172.21.15.112:0/937991424 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7fab380bc3d0 con 0x7fab48101790 2024-04-24T23:12:32.008 INFO:teuthology.orchestra.run.smithi112.stderr:set pool 4 allow_ec_overwrites to true 2024-04-24T23:12:32.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fab300a2b90 msgr2=0x7fab300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fab300a2b90 0x7fab300a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009b60 tx=0x7fab3c005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:32.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 msgr2=0x7fab48197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 0x7fab48197c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab38009230 tx=0x7fab380042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 shutdown_connections 2024-04-24T23:12:32.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fab300a2b90 0x7fab300a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab48103340 0x7fab4819f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab48102980 0x7fab48198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 --2- 172.21.15.112:0/937991424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fab48101790 0x7fab48197c20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 >> 172.21.15.112:0/937991424 conn(0x7fab480fcd20 msgr2=0x7fab48105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:32.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.010+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 shutdown_connections 2024-04-24T23:12:32.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.011+0000 7fab4da5d640 1 -- 172.21.15.112:0/937991424 wait complete. 2024-04-24T23:12:32.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-24T23:12:32.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 -- 172.21.15.112:0/3553709278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 msgr2=0x7fa1d4108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 --2- 172.21.15.112:0/3553709278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 0x7fa1d4108ae0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc009ca0 tx=0x7fa1cc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 -- 172.21.15.112:0/3553709278 shutdown_connections 2024-04-24T23:12:32.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 --2- 172.21.15.112:0/3553709278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 0x7fa1d4108ae0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 --2- 172.21.15.112:0/3553709278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d4101890 0x7fa1d4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 --2- 172.21.15.112:0/3553709278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d41006a0 0x7fa1d4100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 -- 172.21.15.112:0/3553709278 >> 172.21.15.112:0/3553709278 conn(0x7fa1d40fbc30 msgr2=0x7fa1d40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:32.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 -- 172.21.15.112:0/3553709278 shutdown_connections 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 -- 172.21.15.112:0/3553709278 wait complete. 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 Processor -- start 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.140+0000 7fa1dc763640 1 -- start start 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dc763640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d41006a0 0x7fa1d406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dc763640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d4101890 0x7fa1d419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dc763640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 0x7fa1d419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dc763640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410c4f0 con 0x7fa1d4102250 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dc763640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410c370 con 0x7fa1d41006a0 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dc763640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d410c670 con 0x7fa1d4101890 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 0x7fa1d419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 0x7fa1d419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46528/0 (socket says 172.21.15.112:46528) 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 -- 172.21.15.112:0/1029628780 learned_addr learned my addr 172.21.15.112:0/1029628780 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1d9cfe640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d4101890 0x7fa1d419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:32.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1da4ff640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d41006a0 0x7fa1d406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 -- 172.21.15.112:0/1029628780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d4101890 msgr2=0x7fa1d419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d4101890 0x7fa1d419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 -- 172.21.15.112:0/1029628780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d41006a0 msgr2=0x7fa1d406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d41006a0 0x7fa1d406a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 -- 172.21.15.112:0/1029628780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d419f850 con 0x7fa1d4102250 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dad00640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 0x7fa1d419ef40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc031a40 tx=0x7fa1cc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1cb7fe640 1 -- 172.21.15.112:0/1029628780 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc02fb20 con 0x7fa1d4102250 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d419fa80 con 0x7fa1d4102250 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1cb7fe640 1 -- 172.21.15.112:0/1029628780 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa1cc02fcc0 con 0x7fa1d4102250 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1cb7fe640 1 -- 172.21.15.112:0/1029628780 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0468c0 con 0x7fa1d4102250 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.141+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d41a0050 con 0x7fa1d4102250 2024-04-24T23:12:32.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.142+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d4107960 con 0x7fa1d4102250 2024-04-24T23:12:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.144+0000 7fa1cb7fe640 1 -- 172.21.15.112:0/1029628780 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa1cc030be0 con 0x7fa1d4102250 2024-04-24T23:12:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.144+0000 7fa1cb7fe640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1c00a2b90 0x7fa1c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.144+0000 7fa1da4ff640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1c00a2b90 0x7fa1c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.144+0000 7fa1cb7fe640 1 -- 172.21.15.112:0/1029628780 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7fa1cc0f0b50 con 0x7fa1d4102250 2024-04-24T23:12:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.145+0000 7fa1da4ff640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1c00a2b90 0x7fa1c00a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1d41016f0 tx=0x7fa1d0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.148+0000 7fa1cb7fe640 1 -- 172.21.15.112:0/1029628780 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa1cc081740 con 0x7fa1d4102250 2024-04-24T23:12:32.249 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:31 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: 2024-04-24T23:12:31.847+0000 7f8e11d89700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-24T23:12:32.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:32.362+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fa1d402d170 con 0x7fa1d4102250 2024-04-24T23:12:32.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:32 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/937991424' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:12:32.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:32 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/937991424' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:12:32.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:32 smithi112 ceph-mon[29186]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:12:32.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:32 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/937991424' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:12:32.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:32 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/937991424' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:12:32.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:32 smithi150 ceph-mon[32247]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:12:32.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:32 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/937991424' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-24T23:12:32.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:32 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/937991424' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-24T23:12:32.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:32 smithi187 ceph-mon[33292]: osdmap e89: 12 total, 12 up, 12 in 2024-04-24T23:12:33.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.258+0000 7fa1cb7fe640 1 -- 172.21.15.112:0/1029628780 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7fa1cc0bd920 con 0x7fa1d4102250 2024-04-24T23:12:33.260 INFO:teuthology.orchestra.run.smithi112.stderr:added data pool 4 to fsmap 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1c00a2b90 msgr2=0x7fa1c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1c00a2b90 0x7fa1c00a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1d41016f0 tx=0x7fa1d0009110 comp rx=0 tx=0).stop 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 msgr2=0x7fa1d419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 0x7fa1d419ef40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc031a40 tx=0x7fa1cc031ee0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 shutdown_connections 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa1c00a2b90 0x7fa1c00a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d4102250 0x7fa1d419ef40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d4101890 0x7fa1d419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 --2- 172.21.15.112:0/1029628780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa1d41006a0 0x7fa1d406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 >> 172.21.15.112:0/1029628780 conn(0x7fa1d40fbc30 msgr2=0x7fa1d4104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 shutdown_connections 2024-04-24T23:12:33.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.261+0000 7fa1dc763640 1 -- 172.21.15.112:0/1029628780 wait complete. 2024-04-24T23:12:33.276 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 -- 172.21.15.112:0/1657132161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa600102ac0 msgr2=0x7fa600102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 --2- 172.21.15.112:0/1657132161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa600102ac0 0x7fa600102f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009930 tx=0x7fa5e802a420 comp rx=0 tx=0).stop 2024-04-24T23:12:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 -- 172.21.15.112:0/1657132161 shutdown_connections 2024-04-24T23:12:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 --2- 172.21.15.112:0/1657132161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa600103480 0x7fa600109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 --2- 172.21.15.112:0/1657132161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa600102ac0 0x7fa600102f40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 --2- 172.21.15.112:0/1657132161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6000ffaf0 0x7fa6000fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 -- 172.21.15.112:0/1657132161 >> 172.21.15.112:0/1657132161 conn(0x7fa6000fb420 msgr2=0x7fa6000fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 -- 172.21.15.112:0/1657132161 shutdown_connections 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 -- 172.21.15.112:0/1657132161 wait complete. 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.377+0000 7fa6060bd640 1 Processor -- start 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 -- start start 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6000ffaf0 0x7fa600102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa600102ac0 0x7fa600100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa600103480 0x7fa600100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa60010d600 con 0x7fa6000ffaf0 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa60010d480 con 0x7fa600102ac0 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa60010d780 con 0x7fa600103480 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6000ffaf0 0x7fa600102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6000ffaf0 0x7fa600102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46540/0 (socket says 172.21.15.112:46540) 2024-04-24T23:12:33.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 -- 172.21.15.112:0/4125200203 learned_addr learned my addr 172.21.15.112:0/4125200203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5fffff640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa600103480 0x7fa600100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 -- 172.21.15.112:0/4125200203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa600103480 msgr2=0x7fa600100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa600103480 0x7fa600100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 -- 172.21.15.112:0/4125200203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa600102ac0 msgr2=0x7fa600100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5f7fff640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa600102ac0 0x7fa600100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa600102ac0 0x7fa600100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 -- 172.21.15.112:0/4125200203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa600101840 con 0x7fa6000ffaf0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5f7fff640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa600102ac0 0x7fa600100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5ff7fe640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6000ffaf0 0x7fa600102430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c970 tx=0x7fa5f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5fd7fa640 1 -- 172.21.15.112:0/4125200203 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0004220 con 0x7fa6000ffaf0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5fd7fa640 1 -- 172.21.15.112:0/4125200203 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa5f00043c0 con 0x7fa6000ffaf0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6001a1c90 con 0x7fa6000ffaf0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa5fd7fa640 1 -- 172.21.15.112:0/4125200203 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000f6b0 con 0x7fa6000ffaf0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.378+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6001a2190 con 0x7fa6000ffaf0 2024-04-24T23:12:33.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.379+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa600108b20 con 0x7fa6000ffaf0 2024-04-24T23:12:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.381+0000 7fa5fd7fa640 1 -- 172.21.15.112:0/4125200203 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa5f00037a0 con 0x7fa6000ffaf0 2024-04-24T23:12:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.381+0000 7fa5fd7fa640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5ec0a2b90 0x7fa5ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.381+0000 7fa5f7fff640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5ec0a2b90 0x7fa5ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.382+0000 7fa5fd7fa640 1 -- 172.21.15.112:0/4125200203 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa5f00cf0f0 con 0x7fa6000ffaf0 2024-04-24T23:12:33.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.382+0000 7fa5f7fff640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5ec0a2b90 0x7fa5ec0a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80096c0 tx=0x7fa5e8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:33.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.386+0000 7fa5fd7fa640 1 -- 172.21.15.112:0/4125200203 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5f0014030 con 0x7fa6000ffaf0 2024-04-24T23:12:33.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:33 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1029628780' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:12:33.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:33 smithi112 ceph-mon[29186]: pgmap v292: 129 pgs: 31 unknown, 2 creating+peering, 96 active+clean; 578 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-24T23:12:33.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:33 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1029628780' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:12:33.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:33 smithi150 ceph-mon[32247]: pgmap v292: 129 pgs: 31 unknown, 2 creating+peering, 96 active+clean; 578 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-24T23:12:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.600+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa60002d170 con 0x7fa6000ffaf0 2024-04-24T23:12:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.601+0000 7fa5fd7fa640 1 -- 172.21.15.112:0/4125200203 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11812 (secure 0 0 0) 0x7fa60002d170 con 0x7fa6000ffaf0 2024-04-24T23:12:33.603 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:33.603 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:12:32.364408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24524,"qdb_cluster":[24524]},"id":1}]} 2024-04-24T23:12:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5ec0a2b90 msgr2=0x7fa5ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5ec0a2b90 0x7fa5ec0a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80096c0 tx=0x7fa5e8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6000ffaf0 msgr2=0x7fa600102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6000ffaf0 0x7fa600102430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000c970 tx=0x7fa5f000ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 shutdown_connections 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5ec0a2b90 0x7fa5ec0a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa600103480 0x7fa600100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa600102ac0 0x7fa600100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 --2- 172.21.15.112:0/4125200203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6000ffaf0 0x7fa600102430 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 >> 172.21.15.112:0/4125200203 conn(0x7fa6000fb420 msgr2=0x7fa600105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 shutdown_connections 2024-04-24T23:12:33.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.603+0000 7fa6060bd640 1 -- 172.21.15.112:0/4125200203 wait complete. 2024-04-24T23:12:33.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:33.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 -- 172.21.15.112:0/498305854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 msgr2=0x7fb488101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 --2- 172.21.15.112:0/498305854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 0x7fb488101d00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009900 tx=0x7fb47c02a3f0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 -- 172.21.15.112:0/498305854 shutdown_connections 2024-04-24T23:12:33.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 --2- 172.21.15.112:0/498305854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488102240 0x7fb488108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 --2- 172.21.15.112:0/498305854 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 0x7fb488101d00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 --2- 172.21.15.112:0/498305854 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb488100690 0x7fb488100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 -- 172.21.15.112:0/498305854 >> 172.21.15.112:0/498305854 conn(0x7fb4880fbc20 msgr2=0x7fb4880fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 -- 172.21.15.112:0/498305854 shutdown_connections 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 -- 172.21.15.112:0/498305854 wait complete. 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.712+0000 7fb48f49a640 1 Processor -- start 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 -- start start 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb488100690 0x7fb488197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 0x7fb488197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488102240 0x7fb48819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810d550 con 0x7fb488101880 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810d3d0 con 0x7fb488100690 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb48810d6d0 con 0x7fb488102240 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 0x7fb488197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 0x7fb488197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46562/0 (socket says 172.21.15.112:46562) 2024-04-24T23:12:33.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 -- 172.21.15.112:0/3308138428 learned_addr learned my addr 172.21.15.112:0/3308138428 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:33.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48da37640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488102240 0x7fb48819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 -- 172.21.15.112:0/3308138428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488102240 msgr2=0x7fb48819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488102240 0x7fb48819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 -- 172.21.15.112:0/3308138428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb488100690 msgr2=0x7fb488197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48d236640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb488100690 0x7fb488197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb488100690 0x7fb488197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 -- 172.21.15.112:0/3308138428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48819f6a0 con 0x7fb488101880 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48d236640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb488100690 0x7fb488197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48ca35640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 0x7fb488197fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009a20 tx=0x7fb47c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb4867fc640 1 -- 172.21.15.112:0/3308138428 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c03f030 con 0x7fb488101880 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb4867fc640 1 -- 172.21.15.112:0/3308138428 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb47c003dc0 con 0x7fb488101880 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48819f930 con 0x7fb488101880 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb4867fc640 1 -- 172.21.15.112:0/3308138428 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c035070 con 0x7fb488101880 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.713+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48819fe20 con 0x7fb488101880 2024-04-24T23:12:33.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.714+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4881078e0 con 0x7fb488101880 2024-04-24T23:12:33.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.715+0000 7fb4867fc640 1 -- 172.21.15.112:0/3308138428 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb47c042030 con 0x7fb488101880 2024-04-24T23:12:33.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.716+0000 7fb4867fc640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb4700a2b90 0x7fb4700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:33.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.716+0000 7fb48d236640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb4700a2b90 0x7fb4700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:33.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.716+0000 7fb4867fc640 1 -- 172.21.15.112:0/3308138428 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb47c0ef760 con 0x7fb488101880 2024-04-24T23:12:33.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.716+0000 7fb48d236640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb4700a2b90 0x7fb4700a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb4881016e0 tx=0x7fb480005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:33.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.720+0000 7fb4867fc640 1 -- 172.21.15.112:0/3308138428 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb47c0bc480 con 0x7fb488101880 2024-04-24T23:12:33.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:33 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1029628780' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-24T23:12:33.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:33 smithi187 ceph-mon[33292]: pgmap v292: 129 pgs: 31 unknown, 2 creating+peering, 96 active+clean; 578 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 8 op/s 2024-04-24T23:12:33.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.898+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4880623e0 con 0x7fb488101880 2024-04-24T23:12:33.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.899+0000 7fb4867fc640 1 -- 172.21.15.112:0/3308138428 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7fb47c0bbd80 con 0x7fb488101880 2024-04-24T23:12:33.902 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:33.902 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:33.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb4700a2b90 msgr2=0x7fb4700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb4700a2b90 0x7fb4700a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb4881016e0 tx=0x7fb480005db0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 msgr2=0x7fb488197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 0x7fb488197fc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb47c009a20 tx=0x7fb47c004430 comp rx=0 tx=0).stop 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 shutdown_connections 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb4700a2b90 0x7fb4700a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488102240 0x7fb48819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb488101880 0x7fb488197fc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 --2- 172.21.15.112:0/3308138428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb488100690 0x7fb488197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 >> 172.21.15.112:0/3308138428 conn(0x7fb4880fbc20 msgr2=0x7fb488104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 shutdown_connections 2024-04-24T23:12:33.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:33.901+0000 7fb48f49a640 1 -- 172.21.15.112:0/3308138428 wait complete. 2024-04-24T23:12:33.918 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/4132774617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d38101c80 msgr2=0x7f1d381020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/4132774617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d38101c80 0x7f1d381020a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009930 tx=0x7f1d2802a420 comp rx=0 tx=0).stop 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/4132774617 shutdown_connections 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/4132774617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d381012e0 0x7f1d38108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/4132774617 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d38100920 0x7f1d38100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/4132774617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d38101c80 0x7f1d381020a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/4132774617 >> 172.21.15.112:0/4132774617 conn(0x7f1d380fb3b0 msgr2=0x7f1d380fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/4132774617 shutdown_connections 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.017+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/4132774617 wait complete. 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d3ddc1640 1 Processor -- start 2024-04-24T23:12:34.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d3ddc1640 1 -- start start 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d3ddc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d38100920 0x7f1d381003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d3ddc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d381012e0 0x7f1d3819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d3ddc1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d38101c80 0x7f1d3819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d3ddc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3810c4d0 con 0x7f1d381012e0 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d3ddc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3810c350 con 0x7f1d38100920 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d3ddc1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3810c650 con 0x7f1d38101c80 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d36ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d381012e0 0x7f1d3819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d36ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d381012e0 0x7f1d3819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46590/0 (socket says 172.21.15.112:46590) 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d36ffd640 1 -- 172.21.15.112:0/3754881150 learned_addr learned my addr 172.21.15.112:0/3754881150 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d37fff640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d38101c80 0x7f1d3819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d36ffd640 1 -- 172.21.15.112:0/3754881150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d38101c80 msgr2=0x7f1d3819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d36ffd640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d38101c80 0x7f1d3819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d36ffd640 1 -- 172.21.15.112:0/3754881150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d38100920 msgr2=0x7f1d381003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d377fe640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d38100920 0x7f1d381003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d36ffd640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d38100920 0x7f1d381003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d36ffd640 1 -- 172.21.15.112:0/3754881150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3819f8a0 con 0x7f1d381012e0 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d37fff640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d38101c80 0x7f1d3819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.018+0000 7f1d377fe640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d38100920 0x7f1d381003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.019+0000 7f1d36ffd640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d381012e0 0x7f1d3819c970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00c890 tx=0x7f1d2c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.019+0000 7f1d34ff9640 1 -- 172.21.15.112:0/3754881150 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c004220 con 0x7f1d381012e0 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.019+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3819fb90 con 0x7f1d381012e0 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.019+0000 7f1d34ff9640 1 -- 172.21.15.112:0/3754881150 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1d2c0043c0 con 0x7f1d381012e0 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.019+0000 7f1d34ff9640 1 -- 172.21.15.112:0/3754881150 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c004b10 con 0x7f1d381012e0 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.019+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d381a0130 con 0x7f1d381012e0 2024-04-24T23:12:34.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.020+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d38107a00 con 0x7f1d381012e0 2024-04-24T23:12:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.021+0000 7f1d34ff9640 1 -- 172.21.15.112:0/3754881150 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1d2c0051b0 con 0x7f1d381012e0 2024-04-24T23:12:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.022+0000 7f1d34ff9640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1d200a2b90 0x7f1d200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.022+0000 7f1d377fe640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1d200a2b90 0x7f1d200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.022+0000 7f1d34ff9640 1 -- 172.21.15.112:0/3754881150 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1d2c0cf360 con 0x7f1d381012e0 2024-04-24T23:12:34.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.022+0000 7f1d377fe640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1d200a2b90 0x7f1d200a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d280096c0 tx=0x7f1d28005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.026+0000 7f1d34ff9640 1 -- 172.21.15.112:0/3754881150 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d2c014030 con 0x7f1d381012e0 2024-04-24T23:12:34.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.204+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d3802d370 con 0x7f1d381012e0 2024-04-24T23:12:34.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.206+0000 7f1d34ff9640 1 -- 172.21.15.112:0/3754881150 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7f1d2c09c080 con 0x7f1d381012e0 2024-04-24T23:12:34.208 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:34.208 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1d200a2b90 msgr2=0x7f1d200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1d200a2b90 0x7f1d200a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d280096c0 tx=0x7f1d28005be0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d381012e0 msgr2=0x7f1d3819c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d381012e0 0x7f1d3819c970 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00c890 tx=0x7f1d2c00cd50 comp rx=0 tx=0).stop 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 shutdown_connections 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f1d200a2b90 0x7f1d200a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d38101c80 0x7f1d3819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d381012e0 0x7f1d3819c970 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 --2- 172.21.15.112:0/3754881150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1d38100920 0x7f1d381003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 >> 172.21.15.112:0/3754881150 conn(0x7f1d380fb3b0 msgr2=0x7f1d38104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 shutdown_connections 2024-04-24T23:12:34.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.208+0000 7f1d3ddc1640 1 -- 172.21.15.112:0/3754881150 wait complete. 2024-04-24T23:12:34.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/2711115866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88101310 msgr2=0x7f0b88101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/2711115866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88101310 0x7f0b88101790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009ca0 tx=0x7f0b8002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/2711115866 shutdown_connections 2024-04-24T23:12:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/2711115866 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88101310 0x7f0b88101790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/2711115866 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b88100970 0x7f0b88100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/2711115866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b88100010 0x7f0b88100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/2711115866 >> 172.21.15.112:0/2711115866 conn(0x7f0b880fbc30 msgr2=0x7f0b880fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/2711115866 shutdown_connections 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/2711115866 wait complete. 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 Processor -- start 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.332+0000 7f0b8e6bd640 1 -- start start 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8e6bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b88100010 0x7f0b88199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8e6bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88100970 0x7f0b8819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8e6bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b88101310 0x7f0b881a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8e6bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8810f760 con 0x7f0b88100970 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8e6bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8810f5e0 con 0x7f0b88100010 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8e6bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8810f8e0 con 0x7f0b88101310 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88100970 0x7f0b8819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88100970 0x7f0b8819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46608/0 (socket says 172.21.15.112:46608) 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 -- 172.21.15.112:0/78420373 learned_addr learned my addr 172.21.15.112:0/78420373 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8cc5a640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b88101310 0x7f0b881a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 -- 172.21.15.112:0/78420373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b88101310 msgr2=0x7f0b881a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b88101310 0x7f0b881a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 -- 172.21.15.112:0/78420373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b88100010 msgr2=0x7f0b88199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b87fff640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b88100010 0x7f0b88199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b88100010 0x7f0b88199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 -- 172.21.15.112:0/78420373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b881a1890 con 0x7f0b88100970 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b87fff640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b88100010 0x7f0b88199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b877fe640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88100970 0x7f0b8819a1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d770 tx=0x7f0b7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b857fa640 1 -- 172.21.15.112:0/78420373 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c017070 con 0x7f0b88100970 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b881a1b80 con 0x7f0b88100970 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b857fa640 1 -- 172.21.15.112:0/78420373 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0b7c005520 con 0x7f0b88100970 2024-04-24T23:12:34.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b857fa640 1 -- 172.21.15.112:0/78420373 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c010460 con 0x7f0b88100970 2024-04-24T23:12:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.333+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b881a2080 con 0x7f0b88100970 2024-04-24T23:12:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.334+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8810ad60 con 0x7f0b88100970 2024-04-24T23:12:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.336+0000 7f0b857fa640 1 -- 172.21.15.112:0/78420373 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0b7c004ea0 con 0x7f0b88100970 2024-04-24T23:12:34.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.336+0000 7f0b857fa640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0b700a2b90 0x7f0b700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.336+0000 7f0b87fff640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0b700a2b90 0x7f0b700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.336+0000 7f0b857fa640 1 -- 172.21.15.112:0/78420373 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0b7c0ceef0 con 0x7f0b88100970 2024-04-24T23:12:34.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.337+0000 7f0b87fff640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0b700a2b90 0x7f0b700a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b8819b150 tx=0x7f0b78005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.340+0000 7f0b857fa640 1 -- 172.21.15.112:0/78420373 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0b7c09bc10 con 0x7f0b88100970 2024-04-24T23:12:34.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:34 smithi112 ceph-mon[29186]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:12:34.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:34 smithi112 ceph-mon[29186]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:12:34.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:34 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1029628780' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:12:34.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:34 smithi112 ceph-mon[29186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:34.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:34 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4125200203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:34 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3308138428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:34 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3754881150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.554+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b8802d170 con 0x7f0b88100970 2024-04-24T23:12:34.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.555+0000 7f0b857fa640 1 -- 172.21.15.112:0/78420373 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11812 (secure 0 0 0) 0x7f0b7c09b510 con 0x7f0b88100970 2024-04-24T23:12:34.557 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:34.557 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:12:32.364408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24524,"qdb_cluster":[24524]},"id":1}]} 2024-04-24T23:12:34.557 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-04-24T23:12:34.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0b700a2b90 msgr2=0x7f0b700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0b700a2b90 0x7f0b700a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b8819b150 tx=0x7f0b78005db0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88100970 msgr2=0x7f0b8819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88100970 0x7f0b8819a1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d770 tx=0x7f0b7c00dc30 comp rx=0 tx=0).stop 2024-04-24T23:12:34.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 shutdown_connections 2024-04-24T23:12:34.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0b700a2b90 0x7f0b700a5050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b88101310 0x7f0b881a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b88100970 0x7f0b8819a1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 --2- 172.21.15.112:0/78420373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b88100010 0x7f0b88199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 >> 172.21.15.112:0/78420373 conn(0x7f0b880fbc30 msgr2=0x7f0b880fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 shutdown_connections 2024-04-24T23:12:34.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.557+0000 7f0b8e6bd640 1 -- 172.21.15.112:0/78420373 wait complete. 2024-04-24T23:12:34.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:34 smithi150 ceph-mon[32247]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:12:34.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:34 smithi150 ceph-mon[32247]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:12:34.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:34 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1029628780' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:12:34.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:34 smithi150 ceph-mon[32247]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:34.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:34 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4125200203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:34 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3308138428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:34 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3754881150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:34.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/897746602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b981018d0 msgr2=0x7f5b98101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/897746602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b981018d0 0x7f5b98101d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009900 tx=0x7f5b8c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/897746602 shutdown_connections 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/897746602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b98102290 0x7f5b98108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/897746602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b981018d0 0x7f5b98101d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/897746602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b98100bc0 0x7f5b98100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/897746602 >> 172.21.15.112:0/897746602 conn(0x7f5b980fb3b0 msgr2=0x7f5b980fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/897746602 shutdown_connections 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.667+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/897746602 wait complete. 2024-04-24T23:12:34.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b9dc9a640 1 Processor -- start 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b9dc9a640 1 -- start start 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b9dc9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b98100bc0 0x7f5b98197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b9dc9a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b981018d0 0x7f5b98198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b9dc9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b98102290 0x7f5b9819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b9dc9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9810c450 con 0x7f5b98100bc0 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b9dc9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9810c2d0 con 0x7f5b98102290 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b9dc9a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9810c5d0 con 0x7f5b981018d0 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b98100bc0 0x7f5b98197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b977fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b98100bc0 0x7f5b98197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46620/0 (socket says 172.21.15.112:46620) 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b977fe640 1 -- 172.21.15.112:0/353194505 learned_addr learned my addr 172.21.15.112:0/353194505 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b97fff640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b98102290 0x7f5b9819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b977fe640 1 -- 172.21.15.112:0/353194505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b981018d0 msgr2=0x7f5b98198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b977fe640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b981018d0 0x7f5b98198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b977fe640 1 -- 172.21.15.112:0/353194505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b98102290 msgr2=0x7f5b9819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b977fe640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b98102290 0x7f5b9819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.668+0000 7f5b977fe640 1 -- 172.21.15.112:0/353194505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9819f6f0 con 0x7f5b98100bc0 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.669+0000 7f5b977fe640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b98100bc0 0x7f5b98197ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c970 tx=0x7f5b8800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.669+0000 7f5b94ff9640 1 -- 172.21.15.112:0/353194505 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88007c10 con 0x7f5b98100bc0 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.669+0000 7f5b94ff9640 1 -- 172.21.15.112:0/353194505 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5b88007db0 con 0x7f5b98100bc0 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.669+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9819f9e0 con 0x7f5b98100bc0 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.669+0000 7f5b94ff9640 1 -- 172.21.15.112:0/353194505 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800fa40 con 0x7f5b98100bc0 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.669+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9819ff30 con 0x7f5b98100bc0 2024-04-24T23:12:34.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.670+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b981a01c0 con 0x7f5b98100bc0 2024-04-24T23:12:34.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.671+0000 7f5b94ff9640 1 -- 172.21.15.112:0/353194505 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5b8800fbe0 con 0x7f5b98100bc0 2024-04-24T23:12:34.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.671+0000 7f5b94ff9640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b800a2b90 0x7f5b800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.671+0000 7f5b96ffd640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b800a2b90 0x7f5b800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.671+0000 7f5b94ff9640 1 -- 172.21.15.112:0/353194505 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5b880cf710 con 0x7f5b98100bc0 2024-04-24T23:12:34.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.672+0000 7f5b96ffd640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b800a2b90 0x7f5b800a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005c20 tx=0x7f5b8c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.675+0000 7f5b94ff9640 1 -- 172.21.15.112:0/353194505 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5b8809c4c0 con 0x7f5b98100bc0 2024-04-24T23:12:34.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:34 smithi187 ceph-mon[33292]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:12:34.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:34 smithi187 ceph-mon[33292]: osdmap e90: 12 total, 12 up, 12 in 2024-04-24T23:12:34.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:34 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1029628780' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-24T23:12:34.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:34 smithi187 ceph-mon[33292]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:34.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:34 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4125200203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:34 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3308138428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:34 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3754881150' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:34.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.855+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b981a0390 con 0x7f5b98100bc0 2024-04-24T23:12:34.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.857+0000 7f5b94ff9640 1 -- 172.21.15.112:0/353194505 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7f5b8809bdc0 con 0x7f5b98100bc0 2024-04-24T23:12:34.859 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:34.859 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b800a2b90 msgr2=0x7f5b800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b800a2b90 0x7f5b800a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005c20 tx=0x7f5b8c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b98100bc0 msgr2=0x7f5b98197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b98100bc0 0x7f5b98197ad0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800c970 tx=0x7f5b8800ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 shutdown_connections 2024-04-24T23:12:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5b800a2b90 0x7f5b800a5050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b98102290 0x7f5b9819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b981018d0 0x7f5b98198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 --2- 172.21.15.112:0/353194505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b98100bc0 0x7f5b98197ad0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 >> 172.21.15.112:0/353194505 conn(0x7f5b980fb3b0 msgr2=0x7f5b980ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 shutdown_connections 2024-04-24T23:12:34.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.859+0000 7f5b9dc9a640 1 -- 172.21.15.112:0/353194505 wait complete. 2024-04-24T23:12:34.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-24T23:12:27.330443+0000', 'modified': '2024-04-24T23:12:32.364408+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/748900426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 748900426}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 748900426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24524, 'qdb_cluster': [24524]} 2024-04-24T23:12:34.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-24T23:12:34.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:12:34.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.969+0000 7f5407d16640 1 -- 172.21.15.112:0/349446796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 msgr2=0x7f5400100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.969+0000 7f5407d16640 1 --2- 172.21.15.112:0/349446796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 0x7f5400100230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f53fc009ca0 tx=0x7f53fc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.969+0000 7f5407d16640 1 -- 172.21.15.112:0/349446796 shutdown_connections 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.969+0000 7f5407d16640 1 --2- 172.21.15.112:0/349446796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5400101920 0x7f5400108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.969+0000 7f5407d16640 1 --2- 172.21.15.112:0/349446796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5400100f60 0x7f54001013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.969+0000 7f5407d16640 1 --2- 172.21.15.112:0/349446796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 0x7f5400100230 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.969+0000 7f5407d16640 1 -- 172.21.15.112:0/349446796 >> 172.21.15.112:0/349446796 conn(0x7f54000fb380 msgr2=0x7f54000fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 -- 172.21.15.112:0/349446796 shutdown_connections 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 -- 172.21.15.112:0/349446796 wait complete. 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 Processor -- start 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 -- start start 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 0x7f54001097a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5400100f60 0x7f540019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5400101920 0x7f54001a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f540010c490 con 0x7f54000ffe10 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f540010c310 con 0x7f5400100f60 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5407d16640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f540010c610 con 0x7f5400101920 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5405ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 0x7f54001097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5405ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 0x7f54001097a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46644/0 (socket says 172.21.15.112:46644) 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f5405ab2640 1 -- 172.21.15.112:0/699110782 learned_addr learned my addr 172.21.15.112:0/699110782 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.970+0000 7f54062b3640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5400101920 0x7f54001a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f5405ab2640 1 -- 172.21.15.112:0/699110782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5400101920 msgr2=0x7f54001a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f5405ab2640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5400101920 0x7f54001a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f5405ab2640 1 -- 172.21.15.112:0/699110782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5400100f60 msgr2=0x7f540019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f54052b1640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5400100f60 0x7f540019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f5405ab2640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5400100f60 0x7f540019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f5405ab2640 1 -- 172.21.15.112:0/699110782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54001a1a60 con 0x7f54000ffe10 2024-04-24T23:12:34.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f54052b1640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5400100f60 0x7f540019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:34.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f5405ab2640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 0x7f54001097a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f53fc031a40 tx=0x7f53fc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f53f6ffd640 1 -- 172.21.15.112:0/699110782 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc003c30 con 0x7f54000ffe10 2024-04-24T23:12:34.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f53f6ffd640 1 -- 172.21.15.112:0/699110782 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53fc003dd0 con 0x7f54000ffe10 2024-04-24T23:12:34.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f5407d16640 1 -- 172.21.15.112:0/699110782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54001a1c90 con 0x7f54000ffe10 2024-04-24T23:12:34.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f53f6ffd640 1 -- 172.21.15.112:0/699110782 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0047e0 con 0x7f54000ffe10 2024-04-24T23:12:34.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.971+0000 7f5407d16640 1 -- 172.21.15.112:0/699110782 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54001a2180 con 0x7f54000ffe10 2024-04-24T23:12:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.973+0000 7f53f6ffd640 1 -- 172.21.15.112:0/699110782 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f53fc02f710 con 0x7f54000ffe10 2024-04-24T23:12:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f540aa1c740 1 Processor -- start 2024-04-24T23:12:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f540aa1c740 1 -- start start 2024-04-24T23:12:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a387860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a387e30 0x56103a38c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a38cfa0 0x56103a38f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f540aa1c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56103a2eaf00 con 0x56103a387440 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f540aa1c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56103a293750 con 0x56103a38cfa0 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f540aa1c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56103a2e6ec0 con 0x56103a387e30 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f53f6ffd640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53ec0a2c40 0x7f53ec0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a387860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a387860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46648/0 (socket says 172.21.15.112:46648) 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 -- 172.21.15.112:0/1169302157 learned_addr learned my addr 172.21.15.112:0/1169302157 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53ec0a2c40 0x7f53ec0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54052b1640 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a38cfa0 0x56103a38f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f53f6ffd640 1 -- 172.21.15.112:0/699110782 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f53fc0f04d0 con 0x7f54000ffe10 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f5405ab2640 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a387e30 0x56103a38c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 -- 172.21.15.112:0/1169302157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a387e30 msgr2=0x56103a38c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:34.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a387e30 0x56103a38c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 -- 172.21.15.112:0/1169302157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a38cfa0 msgr2=0x56103a38f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a38cfa0 0x56103a38f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54062b3640 1 -- 172.21.15.112:0/1169302157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561039fff2d0 con 0x56103a387440 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.974+0000 7f54052b1640 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a38cfa0 0x56103a38f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.975+0000 7f54062b3640 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a387860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f53f8014c50 tx=0x7f53f80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b180ade22bfe25 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.975+0000 7f54062b3640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53ec0a2c40 0x7f53ec0a5100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f53f8011cb0 tx=0x7f53f8016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.975+0000 7f53ebfff640 1 -- 172.21.15.112:0/1169302157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8014620 con 0x56103a387440 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.975+0000 7f53ebfff640 1 -- 172.21.15.112:0/1169302157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53f800c9c0 con 0x56103a387440 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.975+0000 7f53ebfff640 1 -- 172.21.15.112:0/1169302157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f80047a0 con 0x56103a387440 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- 172.21.15.112:0/1169302157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 msgr2=0x56103a387860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a387860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f53f8014c50 tx=0x7f53f80117b0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- 172.21.15.112:0/1169302157 shutdown_connections 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a38cfa0 0x56103a38f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a387e30 0x56103a38c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1169302157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a387860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- 172.21.15.112:0/1169302157 >> 172.21.15.112:0/1169302157 conn(0x56103a382ab0 msgr2=0x56103a382eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- 172.21.15.112:0/1169302157 shutdown_connections 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- 172.21.15.112:0/1169302157 wait complete. 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- start start 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 Processor -- start 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a391cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a398d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56103a38fef0 con 0x56103a387440 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56103a2a6cf0 con 0x56103a387e30 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56103a390070 con 0x56103a38cfa0 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.976+0000 7f540aa1c740 10 client.0 _ll_register_callbacks cb 0x56103a2a66e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:34.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3917b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46656/0 (socket says 172.21.15.112:46656) 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 -- 172.21.15.112:0/1705306183 learned_addr learned my addr 172.21.15.112:0/1705306183 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f54052b1640 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a391cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 -- 172.21.15.112:0/1705306183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 msgr2=0x56103a398d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a398d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 -- 172.21.15.112:0/1705306183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 msgr2=0x56103a391cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a391cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 -- 172.21.15.112:0/1705306183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103a0c96b0 con 0x56103a387440 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f5405ab2640 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3917b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0edd30 tx=0x7f53fc0bc480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c02bc449f54f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f53e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f53e97fa640 1 -- 172.21.15.112:0/1705306183 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0d5060 con 0x56103a387440 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f53e97fa640 1 -- 172.21.15.112:0/1705306183 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f53fc0b9190 con 0x56103a387440 2024-04-24T23:12:34.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f540aa1c740 1 -- 172.21.15.112:0/1705306183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53f800c9c0 con 0x56103a387440 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f53e97fa640 1 -- 172.21.15.112:0/1705306183 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f53fc0c0ed0 con 0x56103a387440 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f540aa1c740 10 client.14763 fetch_fsmap learned FSMap version 16 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f540aa1c740 1 -- 172.21.15.112:0/1705306183 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x56103a38fef0 con 0x56103a387440 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f53e97fa640 1 -- 172.21.15.112:0/1705306183 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53f800c9c0 con 0x56103a387440 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.977+0000 7f53e97fa640 1 -- 172.21.15.112:0/1705306183 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f53fc004b40 con 0x56103a387440 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f540aa1c740 10 client.14763 fetch_fsmap finished waiting for FSMap version 16 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f540aa1c740 10 client.14763 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f540aa1c740 20 client.14763 populate_metadata read hostname 'smithi112' 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56103a39e3d0 0x56103a39f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f540aa1c740 4 client.14763 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f540aa1c740 1 -- 172.21.15.112:0/1705306183 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56103a2f8440 con 0x56103a39e3d0 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f54052b1640 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56103a39e3d0 0x56103a39f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f54052b1640 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56103a39e3d0 0x56103a39f840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e2cb6bcba77ac2 server_cookie=b4ca2cedb7718d6f in_seq=0 out_seq=0 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.978+0000 7f53e97fa640 10 client.14763 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:12:34.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f53e97fa640 1 -- 172.21.15.112:0/1705306183 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56103a2f8440 con 0x56103a39e3d0 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f53e97fa640 10 client.14763 handle_command_reply: tid=0 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 client.14763 shutdown 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 -- 172.21.15.112:0/1705306183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 msgr2=0x56103a3917b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3917b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0edd30 tx=0x7f53fc0bc480 comp rx=0 tx=0).stop 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 -- 172.21.15.112:0/1705306183 shutdown_connections 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56103a39e3d0 0x56103a39f840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a398d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a391cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 --2- 172.21.15.112:0/1705306183 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3917b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f53e97fa640 0 client.14763 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 -- 172.21.15.112:0/1705306183 >> 172.21.15.112:0/1705306183 conn(0x56103a382ab0 msgr2=0x56103a3818f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 -- 172.21.15.112:0/1705306183 shutdown_connections 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 1 -- 172.21.15.112:0/1705306183 wait complete. 2024-04-24T23:12:34.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.979+0000 7f540aa1c740 20 client.14763 trim_cache size 0 max 16384 2024-04-24T23:12:35.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.997+0000 7f540aa1c740 1 Processor -- start 2024-04-24T23:12:35.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f540aa1c740 1 -- start start 2024-04-24T23:12:35.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3886c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a398cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a3a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f540aa1c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56103a39db80 con 0x56103a387440 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f540aa1c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56103a39e250 con 0x56103a387e30 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f540aa1c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56103a39e920 con 0x56103a38cfa0 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3886c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3886c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46666/0 (socket says 172.21.15.112:46666) 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 -- 172.21.15.112:0/4280507470 learned_addr learned my addr 172.21.15.112:0/4280507470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f54062b3640 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a3a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f54052b1640 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a398cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 -- 172.21.15.112:0/4280507470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 msgr2=0x56103a3a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a3a7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 -- 172.21.15.112:0/4280507470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 msgr2=0x56103a398cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:35.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a398cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 -- 172.21.15.112:0/4280507470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103a387270 con 0x56103a387440 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f54062b3640 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a3a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f54052b1640 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a398cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f5405ab2640 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3886c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0c1090 tx=0x7f53fc0bc5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abef432722891bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.998+0000 7f53eb7fe640 1 -- 172.21.15.112:0/4280507470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a0070 con 0x56103a387440 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f53eb7fe640 1 -- 172.21.15.112:0/4280507470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53fc0bde50 con 0x56103a387440 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f53eb7fe640 1 -- 172.21.15.112:0/4280507470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0c5270 con 0x56103a387440 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 -- 172.21.15.112:0/4280507470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 msgr2=0x56103a3886c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3886c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0c1090 tx=0x7f53fc0bc5b0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 -- 172.21.15.112:0/4280507470 shutdown_connections 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a3a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a398cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 --2- 172.21.15.112:0/4280507470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a3886c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 -- 172.21.15.112:0/4280507470 >> 172.21.15.112:0/4280507470 conn(0x56103a382ab0 msgr2=0x56103a37d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 -- 172.21.15.112:0/4280507470 shutdown_connections 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 -- 172.21.15.112:0/4280507470 wait complete. 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:34.999+0000 7f540aa1c740 1 -- start start 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f540aa1c740 1 Processor -- start 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a394e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a3953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f540aa1c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a397ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f540aa1c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56103a39f6c0 con 0x56103a387440 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f540aa1c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56103a39eff0 con 0x56103a387e30 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f540aa1c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56103a39fd90 con 0x56103a38cfa0 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f540aa1c740 10 client.0 _ll_register_callbacks cb 0x7f53fc0bc8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a394e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a394e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46682/0 (socket says 172.21.15.112:46682) 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 -- 172.21.15.112:0/3733098113 learned_addr learned my addr 172.21.15.112:0/3733098113 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:35.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f54062b3640 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a397ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f54052b1640 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a3953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 -- 172.21.15.112:0/3733098113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 msgr2=0x56103a397ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a397ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 -- 172.21.15.112:0/3733098113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 msgr2=0x56103a3953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a3953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 -- 172.21.15.112:0/3733098113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103a398490 con 0x56103a387440 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f54062b3640 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a397ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f5405ab2640 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a394e80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f53fc02ade0 tx=0x7f53fc0bca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc4c8452160467d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f53e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.000+0000 7f53e97fa640 1 -- 172.21.15.112:0/3733098113 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a0070 con 0x56103a387440 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f53e97fa640 1 -- 172.21.15.112:0/3733098113 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f53fc0c5470 con 0x56103a387440 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 1 -- 172.21.15.112:0/3733098113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53fc0bde50 con 0x56103a387440 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f53e97fa640 1 -- 172.21.15.112:0/3733098113 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f53fc0d5040 con 0x56103a387440 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 10 client.14769 fetch_fsmap learned FSMap version 16 2024-04-24T23:12:35.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 1 -- 172.21.15.112:0/3733098113 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x56103a398490 con 0x56103a387440 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f53e97fa640 1 -- 172.21.15.112:0/3733098113 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53fc0bde50 con 0x56103a387440 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f53e97fa640 1 -- 172.21.15.112:0/3733098113 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f53fc0db200 con 0x56103a387440 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 10 client.14769 fetch_fsmap finished waiting for FSMap version 16 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 10 client.14769 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 20 client.14769 populate_metadata read hostname 'smithi112' 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56103a408c60 0x56103a405770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 4 client.14769 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f540aa1c740 1 -- 172.21.15.112:0/3733098113 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "status"}) -- 0x56103a2f2790 con 0x56103a408c60 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.001+0000 7f54052b1640 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56103a408c60 0x56103a405770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f53e97fa640 10 client.14769 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f54052b1640 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56103a408c60 0x56103a405770 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4edcba493f228b97 server_cookie=3f2bc00525e91630 in_seq=0 out_seq=0 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f53e97fa640 1 -- 172.21.15.112:0/3733098113 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56103a2f2790 con 0x56103a408c60 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f53e97fa640 10 client.14769 handle_command_reply: tid=0 2024-04-24T23:12:35.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 client.14769 shutdown 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 -- 172.21.15.112:0/3733098113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 msgr2=0x56103a394e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a394e80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f53fc02ade0 tx=0x7f53fc0bca20 comp rx=0 tx=0).stop 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 -- 172.21.15.112:0/3733098113 shutdown_connections 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56103a408c60 0x56103a405770 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56103a38cfa0 0x56103a397ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56103a387e30 0x56103a3953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 --2- 172.21.15.112:0/3733098113 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56103a387440 0x56103a394e80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f53e97fa640 0 client.14769 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 -- 172.21.15.112:0/3733098113 >> 172.21.15.112:0/3733098113 conn(0x56103a382ab0 msgr2=0x56103a3886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 -- 172.21.15.112:0/3733098113 shutdown_connections 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 1 -- 172.21.15.112:0/3733098113 wait complete. 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.002+0000 7f540aa1c740 20 client.14769 trim_cache size 0 max 16384 2024-04-24T23:12:35.011 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24524, 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 7.0186743170000003, 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 16, 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 90, 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 13.565245404000001 2024-04-24T23:12:35.012 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 -- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53ec0a2c40 msgr2=0x7f53ec0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53ec0a2c40 0x7f53ec0a5100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f53f8011cb0 tx=0x7f53f8016000 comp rx=0 tx=0).stop 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 -- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 msgr2=0x7f54001097a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 0x7f54001097a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f53fc031a40 tx=0x7f53fc009360 comp rx=0 tx=0).stop 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 -- 172.21.15.112:0/699110782 shutdown_connections 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53ec0a2c40 0x7f53ec0a5100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5400101920 0x7f54001a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5400100f60 0x7f540019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 --2- 172.21.15.112:0/699110782 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54000ffe10 0x7f54001097a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 -- 172.21.15.112:0/699110782 >> 172.21.15.112:0/699110782 conn(0x7f54000fb380 msgr2=0x7f5400104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 -- 172.21.15.112:0/699110782 shutdown_connections 2024-04-24T23:12:35.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:35.003+0000 7f5407d16640 1 -- 172.21.15.112:0/699110782 wait complete. 2024-04-24T23:12:35.016 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:12:35.016 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:12:35.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:12:35.251 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:35 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/78420373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:35 smithi112 ceph-mon[29186]: pgmap v294: 129 pgs: 20 unknown, 109 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-24T23:12:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:35 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/353194505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:35.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:35 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/78420373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:35.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:35 smithi150 ceph-mon[32247]: pgmap v294: 129 pgs: 20 unknown, 109 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-24T23:12:35.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:35 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/353194505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:35.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:35 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/78420373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:35.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:35 smithi187 ceph-mon[33292]: pgmap v294: 129 pgs: 20 unknown, 109 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-04-24T23:12:35.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:35 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/353194505' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:36.416 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:36 smithi112 ceph-mon[29186]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:12:36.416 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:36 smithi112 ceph-mon[29186]: Cluster is now healthy 2024-04-24T23:12:36.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:36 smithi150 ceph-mon[32247]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:12:36.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:36 smithi150 ceph-mon[32247]: Cluster is now healthy 2024-04-24T23:12:36.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:36 smithi187 ceph-mon[33292]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:12:36.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:36 smithi187 ceph-mon[33292]: Cluster is now healthy 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.959+0000 7f86c39a4700 1 -- 172.21.15.112:0/1033837578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc1071b0 msgr2=0x7f86bc107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.959+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1033837578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc1071b0 0x7f86bc107590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009990 tx=0x7f86ac009ca0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.959+0000 7f86c39a4700 1 -- 172.21.15.112:0/1033837578 shutdown_connections 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.959+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1033837578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86bc101b10 0x7f86bc10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.959+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1033837578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc1071b0 0x7f86bc107590 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.959+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1033837578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc101170 0x7f86bc1015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.959+0000 7f86c39a4700 1 -- 172.21.15.112:0/1033837578 >> 172.21.15.112:0/1033837578 conn(0x7f86bc078060 msgr2=0x7f86bc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 -- 172.21.15.112:0/1033837578 shutdown_connections 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 -- 172.21.15.112:0/1033837578 wait complete. 2024-04-24T23:12:36.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 Processor -- start 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 -- start start 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc101170 0x7f86bc19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc101b10 0x7f86bc19dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86bc1071b0 0x7f86bc1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc111630 con 0x7f86bc101170 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1114b0 con 0x7f86bc1071b0 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c39a4700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1117b0 con 0x7f86bc101b10 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c1740700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc101170 0x7f86bc19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c1740700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc101170 0x7f86bc19d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46702/0 (socket says 172.21.15.112:46702) 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.960+0000 7f86c1740700 1 -- 172.21.15.112:0/1526457472 learned_addr learned my addr 172.21.15.112:0/1526457472 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86c1740700 1 -- 172.21.15.112:0/1526457472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc101b10 msgr2=0x7f86bc19dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86c1740700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc101b10 0x7f86bc19dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86c1740700 1 -- 172.21.15.112:0/1526457472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86bc1071b0 msgr2=0x7f86bc1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86c1740700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86bc1071b0 0x7f86bc1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86c1740700 1 -- 172.21.15.112:0/1526457472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc198010 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86c1740700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc101170 0x7f86bc19d6f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86ac000c00 tx=0x7f86ac00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86b27fc700 1 -- 172.21.15.112:0/1526457472 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac027070 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86b27fc700 1 -- 172.21.15.112:0/1526457472 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f86ac00fc30 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc1982a0 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86b27fc700 1 -- 172.21.15.112:0/1526457472 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0177b0 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.961+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1a45c0 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.962+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc102860 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.963+0000 7f86b27fc700 1 -- 172.21.15.112:0/1526457472 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f86ac02a070 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.963+0000 7f86b27fc700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f86a80a2c20 0x7f86a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.964+0000 7f86c0f3f700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f86a80a2c20 0x7f86a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.964+0000 7f86b27fc700 1 -- 172.21.15.112:0/1526457472 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f86ac0d0150 con 0x7f86bc101170 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.964+0000 7f86c0f3f700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f86a80a2c20 0x7f86a80a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0786d0 tx=0x7f86b8006d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:36.968+0000 7f86b27fc700 1 -- 172.21.15.112:0/1526457472 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f86ac013070 con 0x7f86bc101170 2024-04-24T23:12:37.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.209+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f86bc02db30 con 0x7f86bc101170 2024-04-24T23:12:37.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.210+0000 7f86b27fc700 1 -- 172.21.15.112:0/1526457472 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f86ac09cf00 con 0x7f86bc101170 2024-04-24T23:12:37.212 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f86a80a2c20 msgr2=0x7f86a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f86a80a2c20 0x7f86a80a50e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0786d0 tx=0x7f86b8006d20 comp rx=0 tx=0).stop 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc101170 msgr2=0x7f86bc19d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc101170 0x7f86bc19d6f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86ac000c00 tx=0x7f86ac00f740 comp rx=0 tx=0).stop 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 shutdown_connections 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f86a80a2c20 0x7f86a80a50e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86bc1071b0 0x7f86bc1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86bc101170 0x7f86bc19d6f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 --2- 172.21.15.112:0/1526457472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86bc101b10 0x7f86bc19dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 >> 172.21.15.112:0/1526457472 conn(0x7f86bc078060 msgr2=0x7f86bc0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 shutdown_connections 2024-04-24T23:12:37.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37.212+0000 7f86c39a4700 1 -- 172.21.15.112:0/1526457472 wait complete. 2024-04-24T23:12:37.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:37 smithi112 ceph-mon[29186]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-04-24T23:12:37.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:37 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1526457472' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:37.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:37 smithi150 ceph-mon[32247]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-04-24T23:12:37.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:37 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1526457472' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:37.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:37 smithi187 ceph-mon[33292]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 4 op/s 2024-04-24T23:12:37.736 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:37 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1526457472' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:12:37.892 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T23:12:37.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:37.893 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-24T23:12:37.908 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-24T23:12:37.909 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:12:37.910 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:37.910 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:12:37.910 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:12:37.910 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:12:37.910 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42233sec preferred_lft 42233sec 2024-04-24T23:12:37.910 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5787:d070:61ba:675f/64 scope link noprefixroute 2024-04-24T23:12:37.910 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:12:37.910 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:12:37.911 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:37.911 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:12:37.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:12:37.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-04-24T23:12:37.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-04-24T23:12:37.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:12:37.911 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:12:37.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:38.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:38.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:38.086 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:12:38.109 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-24T23:12:38.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:38.111 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-24T23:12:38.179 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-24T23:12:38.179 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:12:38.180 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:12:38.180 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:12:38.180 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:12:38.180 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:38.180 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:12:38.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:12:38.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:12:38.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:12:38.181 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:12:38.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:38.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:38.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:38.442 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-24T23:12:38.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:38.457 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-04-24T23:12:38.512 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:38.512 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:12:38.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:12:38.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-24T23:12:38.512 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:12:38.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:38.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:38.627 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-24T23:12:38.628 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:38.628 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:12:38.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-24T23:12:38.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:12:38.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-24T23:12:38.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-24T23:12:38.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-24T23:12:38.629 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:12:38.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:38.772 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:12:38.772 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:12:38.772 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-04-24T23:12:38.772 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:12:38.772 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:12:38.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:12:38.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:38.848 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-24T23:12:38.848 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:12:38.848 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:12:38.862 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-24T23:12:38.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:12:38.863 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:12:38.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-24T23:12:38.982 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:12:39.038 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-24T23:12:39.038 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-24T23:12:39.039 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-24T23:12:39.040 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/66fabafbbb504207a4a868f4413b3848b617d90f271f781bad868745165b8f92/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/61eef271c704b46a49519939a3b0859f7ee90732b3d952ec5fb74555eab85fe4/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/0 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/583e0d7ce37c130110e0cbee6e74ac742bc9fda75dfde954aec02b9dc5ac7db9/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/85551b41fa8d401de21c58ba137a31ae8acb1f75cf9f83997f80d542559b6aab/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/ac84306431aa27f19994432dc14208a0a0d9d30b7607ff62833c2665f6b00367/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/01e292f0b6c520ec6d2a654994154b711c4f8a96da0e47c548f80fee528b90c7/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/48ffd86b76c8f27f03f5a746973a13f986c7848fe68a7a10ee78979d3bbc1f41/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/85f088d41c32276c12caf8fe7c0687222134396a88a451d31be4440a3b503cf4/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/ca813cd0be7dbae0e9476301151dd64c45ea898b71d928c5ffe93de09018a53e/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/a853ce951bbb95b86f2852bce37d9b56087d4cfdfcf7f2ce9f2f5934e4e4a8c6/merged 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-24T23:12:39.041 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:12:39.042 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:12:39.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:39.042 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-24T23:12:39.094 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T23:12:39.094 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-24T23:12:39.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-24T23:12:39.159 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:12:39.215 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-24T23:12:39.215 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-24T23:12:39.215 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-24T23:12:39.215 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-24T23:12:39.215 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-24T23:12:39.215 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-24T23:12:39.216 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/66fabafbbb504207a4a868f4413b3848b617d90f271f781bad868745165b8f92/merged 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/61eef271c704b46a49519939a3b0859f7ee90732b3d952ec5fb74555eab85fe4/merged 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/0 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/583e0d7ce37c130110e0cbee6e74ac742bc9fda75dfde954aec02b9dc5ac7db9/merged 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/85551b41fa8d401de21c58ba137a31ae8acb1f75cf9f83997f80d542559b6aab/merged 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/ac84306431aa27f19994432dc14208a0a0d9d30b7607ff62833c2665f6b00367/merged 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/01e292f0b6c520ec6d2a654994154b711c4f8a96da0e47c548f80fee528b90c7/merged 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/48ffd86b76c8f27f03f5a746973a13f986c7848fe68a7a10ee78979d3bbc1f41/merged 2024-04-24T23:12:39.217 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/85f088d41c32276c12caf8fe7c0687222134396a88a451d31be4440a3b503cf4/merged 2024-04-24T23:12:39.218 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/ca813cd0be7dbae0e9476301151dd64c45ea898b71d928c5ffe93de09018a53e/merged 2024-04-24T23:12:39.218 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/containers/storage/overlay/a853ce951bbb95b86f2852bce37d9b56087d4cfdfcf7f2ce9f2f5934e4e4a8c6/merged 2024-04-24T23:12:39.218 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-24T23:12:39.218 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:12:39.218 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:12:39.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:39.218 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-24T23:12:39.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:2024-04-24T23:12:39.222+0000 7f9830f20480 -1 init, newargv = 0x558b6b2c6130 newargc=15 2024-04-24T23:12:39.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[73878]: starting ceph client 2024-04-24T23:12:39.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[73878]: starting fuse 2024-04-24T23:12:39.238 INFO:teuthology.orchestra.run.smithi112.stdout:115 2024-04-24T23:12:39.238 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> import re 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> import os 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2024-04-24T23:12:39.239 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> return f 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.admin")) 2024-04-24T23:12:39.240 DEBUG:teuthology.orchestra.run.smithi112:> ' 2024-04-24T23:12:39.355 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.admin.73878.asok 2024-04-24T23:12:39.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:39.365 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73878.asok 2024-04-24T23:12:39.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:39.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73878.asok status 2024-04-24T23:12:39.499 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:12:39.499 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2024-04-24T23:12:39.499 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "admin", 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "73878", 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 14781, 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-04-24T23:12:39.500 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 14781 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3464006955 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3464006955 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.14781 192.168.0.1:0/3464006955", 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/3464006955", 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2024-04-24T23:12:39.501 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 16, 2024-04-24T23:12:39.502 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 90, 2024-04-24T23:12:39.502 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2024-04-24T23:12:39.502 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2024-04-24T23:12:39.502 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2024-04-24T23:12:39.502 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:12:39.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:39.508 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2024-04-24T23:12:39.508 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:12:39.523 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2024-04-24T23:12:39.523 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:12:39.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:39.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:12:39.588 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-24T23:12:39.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:12:39.698 INFO:teuthology.orchestra.run.smithi112.stdout:# file: . 2024-04-24T23:12:39.698 INFO:teuthology.orchestra.run.smithi112.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-24T23:12:39.698 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:39.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:12:39.704 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:12:39.704 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:12:39.720 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:39.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 -- 172.21.15.112:0/1088386730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 msgr2=0x7f585c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 --2- 172.21.15.112:0/1088386730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 0x7f585c100aa0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5854009ca0 tx=0x7f585402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 -- 172.21.15.112:0/1088386730 shutdown_connections 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 --2- 172.21.15.112:0/1088386730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585c102230 0x7f585c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 --2- 172.21.15.112:0/1088386730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f585c101870 0x7f585c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 --2- 172.21.15.112:0/1088386730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 0x7f585c100aa0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 -- 172.21.15.112:0/1088386730 >> 172.21.15.112:0/1088386730 conn(0x7f585c0fbbf0 msgr2=0x7f585c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 -- 172.21.15.112:0/1088386730 shutdown_connections 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.857+0000 7f5861d48640 1 -- 172.21.15.112:0/1088386730 wait complete. 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f5861d48640 1 Processor -- start 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f5861d48640 1 -- start start 2024-04-24T23:12:39.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f5861d48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 0x7f585c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f5861d48640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f585c101870 0x7f585c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f5861d48640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585c102230 0x7f585c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f5861d48640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10c490 con 0x7f585c100680 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f5861d48640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10c310 con 0x7f585c101870 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f5861d48640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f585c10c610 con 0x7f585c102230 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f585b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 0x7f585c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f585b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 0x7f585c069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46732/0 (socket says 172.21.15.112:46732) 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.858+0000 7f585b7fe640 1 -- 172.21.15.112:0/4233934631 learned_addr learned my addr 172.21.15.112:0/4233934631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f585affd640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f585c101870 0x7f585c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f585b7fe640 1 -- 172.21.15.112:0/4233934631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585c102230 msgr2=0x7f585c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f585b7fe640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585c102230 0x7f585c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f585b7fe640 1 -- 172.21.15.112:0/4233934631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f585c101870 msgr2=0x7f585c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:39.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f585b7fe640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f585c101870 0x7f585c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f585b7fe640 1 -- 172.21.15.112:0/4233934631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c19f810 con 0x7f585c100680 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f585affd640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f585c101870 0x7f585c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f585b7fe640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 0x7f585c069460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5854000f80 tx=0x7f585402f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f5858ff9640 1 -- 172.21.15.112:0/4233934631 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585402fdc0 con 0x7f585c100680 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f5858ff9640 1 -- 172.21.15.112:0/4233934631 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5854031df0 con 0x7f585c100680 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c19faa0 con 0x7f585c100680 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f5858ff9640 1 -- 172.21.15.112:0/4233934631 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58540469e0 con 0x7f585c100680 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.859+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c1a0070 con 0x7f585c100680 2024-04-24T23:12:39.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.860+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c1079a0 con 0x7f585c100680 2024-04-24T23:12:39.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.861+0000 7f5858ff9640 1 -- 172.21.15.112:0/4233934631 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5854046b80 con 0x7f585c100680 2024-04-24T23:12:39.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.861+0000 7f5858ff9640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f58480a2b90 0x7f58480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:39.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.862+0000 7f585affd640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f58480a2b90 0x7f58480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:39.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.862+0000 7f5858ff9640 1 -- 172.21.15.112:0/4233934631 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f58540f0aa0 con 0x7f585c100680 2024-04-24T23:12:39.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.862+0000 7f585affd640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f58480a2b90 0x7f58480a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f585c0691d0 tx=0x7f5844005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:39.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:39.865+0000 7f5858ff9640 1 -- 172.21.15.112:0/4233934631 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f58540bd7c0 con 0x7f585c100680 2024-04-24T23:12:40.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.048+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f585c02d3e0 con 0x7f585c100680 2024-04-24T23:12:40.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.050+0000 7f5858ff9640 1 -- 172.21.15.112:0/4233934631 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7f58540bd0c0 con 0x7f585c100680 2024-04-24T23:12:40.052 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:40.052 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f58480a2b90 msgr2=0x7f58480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f58480a2b90 0x7f58480a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f585c0691d0 tx=0x7f5844005e50 comp rx=0 tx=0).stop 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 msgr2=0x7f585c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 0x7f585c069460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5854000f80 tx=0x7f585402f840 comp rx=0 tx=0).stop 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 shutdown_connections 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f58480a2b90 0x7f58480a5050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f585c102230 0x7f585c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f585c101870 0x7f585c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 --2- 172.21.15.112:0/4233934631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f585c100680 0x7f585c069460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 >> 172.21.15.112:0/4233934631 conn(0x7f585c0fbbf0 msgr2=0x7f585c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 shutdown_connections 2024-04-24T23:12:40.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.052+0000 7f5861d48640 1 -- 172.21.15.112:0/4233934631 wait complete. 2024-04-24T23:12:40.066 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:40.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:40.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 -- 172.21.15.112:0/1297536354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 msgr2=0x7fce0c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 --2- 172.21.15.112:0/1297536354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 0x7fce0c100a90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009900 tx=0x7fcdfc02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 -- 172.21.15.112:0/1297536354 shutdown_connections 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 --2- 172.21.15.112:0/1297536354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0c102220 0x7fce0c108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 --2- 172.21.15.112:0/1297536354 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c101860 0x7fce0c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 --2- 172.21.15.112:0/1297536354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 0x7fce0c100a90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 -- 172.21.15.112:0/1297536354 >> 172.21.15.112:0/1297536354 conn(0x7fce0c0fbbe0 msgr2=0x7fce0c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 -- 172.21.15.112:0/1297536354 shutdown_connections 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.166+0000 7fce119c8640 1 -- 172.21.15.112:0/1297536354 wait complete. 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce119c8640 1 Processor -- start 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce119c8640 1 -- start start 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce119c8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 0x7fce0c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce119c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0c101860 0x7fce0c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce119c8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c102220 0x7fce0c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce119c8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c10d530 con 0x7fce0c100670 2024-04-24T23:12:40.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce119c8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c10d3b0 con 0x7fce0c102220 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce119c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c10d6b0 con 0x7fce0c101860 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 0x7fce0c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 0x7fce0c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46758/0 (socket says 172.21.15.112:46758) 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 -- 172.21.15.112:0/671578378 learned_addr learned my addr 172.21.15.112:0/671578378 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce03fff640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0c101860 0x7fce0c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0b7fe640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c102220 0x7fce0c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 -- 172.21.15.112:0/671578378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0c101860 msgr2=0x7fce0c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0c101860 0x7fce0c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 -- 172.21.15.112:0/671578378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c102220 msgr2=0x7fce0c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c102220 0x7fce0c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 -- 172.21.15.112:0/671578378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c19f7b0 con 0x7fce0c100670 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce03fff640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0c101860 0x7fce0c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce0affd640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 0x7fce0c197b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc006010 tx=0x7fcdfc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.167+0000 7fce109c6640 1 -- 172.21.15.112:0/671578378 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc03f030 con 0x7fce0c100670 2024-04-24T23:12:40.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.168+0000 7fce109c6640 1 -- 172.21.15.112:0/671578378 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcdfc003d00 con 0x7fce0c100670 2024-04-24T23:12:40.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.168+0000 7fce109c6640 1 -- 172.21.15.112:0/671578378 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc035070 con 0x7fce0c100670 2024-04-24T23:12:40.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.168+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c19fa40 con 0x7fce0c100670 2024-04-24T23:12:40.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.168+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c19ff30 con 0x7fce0c100670 2024-04-24T23:12:40.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.169+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c107930 con 0x7fce0c100670 2024-04-24T23:12:40.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.170+0000 7fce109c6640 1 -- 172.21.15.112:0/671578378 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcdfc042030 con 0x7fce0c100670 2024-04-24T23:12:40.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.170+0000 7fce109c6640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcdf80a2b90 0x7fcdf80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.170+0000 7fce03fff640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcdf80a2b90 0x7fcdf80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.170+0000 7fce109c6640 1 -- 172.21.15.112:0/671578378 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcdfc0efa20 con 0x7fce0c100670 2024-04-24T23:12:40.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.171+0000 7fce03fff640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcdf80a2b90 0x7fcdf80a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce0c1990a0 tx=0x7fcdf400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.175+0000 7fce109c6640 1 -- 172.21.15.112:0/671578378 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcdfc0bc740 con 0x7fce0c100670 2024-04-24T23:12:40.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:39 smithi187 ceph-mon[33292]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 131 B/s wr, 0 op/s 2024-04-24T23:12:40.271 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:39 smithi150 ceph-mon[32247]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 131 B/s wr, 0 op/s 2024-04-24T23:12:40.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:39 smithi112 ceph-mon[29186]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 131 B/s wr, 0 op/s 2024-04-24T23:12:40.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.350+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fce0c02d530 con 0x7fce0c100670 2024-04-24T23:12:40.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.350+0000 7fce109c6640 1 -- 172.21.15.112:0/671578378 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fcdfc0bc040 con 0x7fce0c100670 2024-04-24T23:12:40.352 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:12:40.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcdf80a2b90 msgr2=0x7fcdf80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcdf80a2b90 0x7fcdf80a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce0c1990a0 tx=0x7fcdf400a5c0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 msgr2=0x7fce0c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 0x7fce0c197b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc006010 tx=0x7fcdfc004430 comp rx=0 tx=0).stop 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 shutdown_connections 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcdf80a2b90 0x7fcdf80a5050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce0c102220 0x7fce0c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce0c101860 0x7fce0c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 --2- 172.21.15.112:0/671578378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0c100670 0x7fce0c197b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 >> 172.21.15.112:0/671578378 conn(0x7fce0c0fbbe0 msgr2=0x7fce0c104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 shutdown_connections 2024-04-24T23:12:40.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.352+0000 7fce119c8640 1 -- 172.21.15.112:0/671578378 wait complete. 2024-04-24T23:12:40.365 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:40.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2834822447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 msgr2=0x7fa544100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2834822447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 0x7fa544100ac0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa540009900 tx=0x7fa54002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:40.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2834822447 shutdown_connections 2024-04-24T23:12:40.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2834822447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa544102250 0x7fa544108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2834822447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544101890 0x7fa544101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2834822447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 0x7fa544100ac0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2834822447 >> 172.21.15.112:0/2834822447 conn(0x7fa5440fbc30 msgr2=0x7fa5440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2834822447 shutdown_connections 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.464+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2834822447 wait complete. 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54ba9a640 1 Processor -- start 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54ba9a640 1 -- start start 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54ba9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 0x7fa544104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54ba9a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa544101890 0x7fa544102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54ba9a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544102250 0x7fa5441031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54ba9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa54410c450 con 0x7fa5441006a0 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54ba9a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa54410c2d0 con 0x7fa544101890 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54ba9a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa54410c5d0 con 0x7fa544102250 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 0x7fa544104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 0x7fa544104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46766/0 (socket says 172.21.15.112:46766) 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 -- 172.21.15.112:0/2355934895 learned_addr learned my addr 172.21.15.112:0/2355934895 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549035640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa544101890 0x7fa544102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa54a037640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544102250 0x7fa5441031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 -- 172.21.15.112:0/2355934895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544102250 msgr2=0x7fa5441031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544102250 0x7fa5441031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 -- 172.21.15.112:0/2355934895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa544101890 msgr2=0x7fa544102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa544101890 0x7fa544102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 -- 172.21.15.112:0/2355934895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa544103a40 con 0x7fa5441006a0 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.465+0000 7fa549836640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 0x7fa544104630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa540009230 tx=0x7fa5400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.466+0000 7fa53affd640 1 -- 172.21.15.112:0/2355934895 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54003f030 con 0x7fa5441006a0 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.466+0000 7fa53affd640 1 -- 172.21.15.112:0/2355934895 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa540003d00 con 0x7fa5441006a0 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.466+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5441a1ca0 con 0x7fa5441006a0 2024-04-24T23:12:40.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.466+0000 7fa53affd640 1 -- 172.21.15.112:0/2355934895 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa540035070 con 0x7fa5441006a0 2024-04-24T23:12:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.466+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5441a21a0 con 0x7fa5441006a0 2024-04-24T23:12:40.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.467+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5441a2370 con 0x7fa5441006a0 2024-04-24T23:12:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.468+0000 7fa53affd640 1 -- 172.21.15.112:0/2355934895 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa540042030 con 0x7fa5441006a0 2024-04-24T23:12:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.468+0000 7fa53affd640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5300a2b90 0x7fa5300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.468+0000 7fa549035640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5300a2b90 0x7fa5300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.468+0000 7fa53affd640 1 -- 172.21.15.112:0/2355934895 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa5400ef770 con 0x7fa5441006a0 2024-04-24T23:12:40.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.469+0000 7fa549035640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5300a2b90 0x7fa5300a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5441043a0 tx=0x7fa53400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.473+0000 7fa53affd640 1 -- 172.21.15.112:0/2355934895 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5400bbe40 con 0x7fa5441006a0 2024-04-24T23:12:40.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.646+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa5441a2540 con 0x7fa5441006a0 2024-04-24T23:12:40.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.646+0000 7fa53affd640 1 -- 172.21.15.112:0/2355934895 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fa5400c0140 con 0x7fa5441006a0 2024-04-24T23:12:40.648 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5300a2b90 msgr2=0x7fa5300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5300a2b90 0x7fa5300a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5441043a0 tx=0x7fa53400b390 comp rx=0 tx=0).stop 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 msgr2=0x7fa544104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 0x7fa544104630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa540009230 tx=0x7fa5400042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 shutdown_connections 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5300a2b90 0x7fa5300a5050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa544102250 0x7fa5441031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa544101890 0x7fa544102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 --2- 172.21.15.112:0/2355934895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5441006a0 0x7fa544104630 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 >> 172.21.15.112:0/2355934895 conn(0x7fa5440fbc30 msgr2=0x7fa5440fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 shutdown_connections 2024-04-24T23:12:40.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.648+0000 7fa54ba9a640 1 -- 172.21.15.112:0/2355934895 wait complete. 2024-04-24T23:12:40.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:40.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.759+0000 7faad162c640 1 -- 172.21.15.112:0/2423526667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 msgr2=0x7faacc102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.759+0000 7faad162c640 1 --2- 172.21.15.112:0/2423526667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 0x7faacc102f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faac0009930 tx=0x7faac002a420 comp rx=0 tx=0).stop 2024-04-24T23:12:40.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.759+0000 7faad162c640 1 -- 172.21.15.112:0/2423526667 shutdown_connections 2024-04-24T23:12:40.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.759+0000 7faad162c640 1 --2- 172.21.15.112:0/2423526667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc1034a0 0x7faacc109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 --2- 172.21.15.112:0/2423526667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 0x7faacc102f60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 --2- 172.21.15.112:0/2423526667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faacc0ffa70 0x7faacc0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 -- 172.21.15.112:0/2423526667 >> 172.21.15.112:0/2423526667 conn(0x7faacc0fb380 msgr2=0x7faacc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 -- 172.21.15.112:0/2423526667 shutdown_connections 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 -- 172.21.15.112:0/2423526667 wait complete. 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 Processor -- start 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 -- start start 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc0ffa70 0x7faacc102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 0x7faacc100a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faacc1034a0 0x7faacc100fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faacc10d5b0 con 0x7faacc102ae0 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faacc10d430 con 0x7faacc1034a0 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.760+0000 7faad162c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faacc10d730 con 0x7faacc0ffa70 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 0x7faacc100a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 0x7faacc100a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34740/0 (socket says 172.21.15.112:34740) 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 -- 172.21.15.112:0/3720966757 learned_addr learned my addr 172.21.15.112:0/3720966757 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:40.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faacb7fe640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faacc1034a0 0x7faacc100fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faacaffd640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc0ffa70 0x7faacc102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 -- 172.21.15.112:0/3720966757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc0ffa70 msgr2=0x7faacc102450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc0ffa70 0x7faacc102450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 -- 172.21.15.112:0/3720966757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faacc1034a0 msgr2=0x7faacc100fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faacc1034a0 0x7faacc100fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 -- 172.21.15.112:0/3720966757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc101860 con 0x7faacc102ae0 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faacb7fe640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faacc1034a0 0x7faacc100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faaca7fc640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 0x7faacc100a90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faac0031570 tx=0x7faac00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faab3fff640 1 -- 172.21.15.112:0/3720966757 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac003f030 con 0x7faacc102ae0 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faacc1a1c80 con 0x7faacc102ae0 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faab3fff640 1 -- 172.21.15.112:0/3720966757 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faac0003d00 con 0x7faacc102ae0 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faab3fff640 1 -- 172.21.15.112:0/3720966757 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0035070 con 0x7faacc102ae0 2024-04-24T23:12:40.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.761+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faacc1a21d0 con 0x7faacc102ae0 2024-04-24T23:12:40.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.762+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faacc108c10 con 0x7faacc102ae0 2024-04-24T23:12:40.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.763+0000 7faab3fff640 1 -- 172.21.15.112:0/3720966757 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7faac0042030 con 0x7faacc102ae0 2024-04-24T23:12:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.764+0000 7faab3fff640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faab40a2b90 0x7faab40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.764+0000 7faab3fff640 1 -- 172.21.15.112:0/3720966757 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7faac00ef820 con 0x7faacc102ae0 2024-04-24T23:12:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.764+0000 7faacaffd640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faab40a2b90 0x7faab40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:40.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.764+0000 7faacaffd640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faab40a2b90 0x7faab40a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faacc069b00 tx=0x7faabc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:40.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.769+0000 7faab3fff640 1 -- 172.21.15.112:0/3720966757 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faac00bc540 con 0x7faacc102ae0 2024-04-24T23:12:40.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.946+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7faacc0623e0 con 0x7faacc102ae0 2024-04-24T23:12:40.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.947+0000 7faab3fff640 1 -- 172.21.15.112:0/3720966757 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7faac00bbe40 con 0x7faacc102ae0 2024-04-24T23:12:40.949 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:12:40.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faab40a2b90 msgr2=0x7faab40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faab40a2b90 0x7faab40a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7faacc069b00 tx=0x7faabc005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:40.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 msgr2=0x7faacc100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:40.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 0x7faacc100a90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faac0031570 tx=0x7faac00042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 shutdown_connections 2024-04-24T23:12:40.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faab40a2b90 0x7faab40a5050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faacc1034a0 0x7faacc100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faacc102ae0 0x7faacc100a90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 --2- 172.21.15.112:0/3720966757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faacc0ffa70 0x7faacc102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:40.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 >> 172.21.15.112:0/3720966757 conn(0x7faacc0fb380 msgr2=0x7faacc105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:40.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 shutdown_connections 2024-04-24T23:12:40.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:40.949+0000 7faad162c640 1 -- 172.21.15.112:0/3720966757 wait complete. 2024-04-24T23:12:40.960 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:40 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4233934631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:40.960 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:40 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/671578378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:40.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:40 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2355934895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:40.962 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 -- 172.21.15.112:0/166518126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 msgr2=0x7ff00c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 --2- 172.21.15.112:0/166518126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 0x7ff00c101d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff000009900 tx=0x7ff00002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 -- 172.21.15.112:0/166518126 shutdown_connections 2024-04-24T23:12:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 --2- 172.21.15.112:0/166518126 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c102250 0x7ff00c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 --2- 172.21.15.112:0/166518126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 0x7ff00c101d10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 --2- 172.21.15.112:0/166518126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c1006a0 0x7ff00c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 -- 172.21.15.112:0/166518126 >> 172.21.15.112:0/166518126 conn(0x7ff00c0fbc10 msgr2=0x7ff00c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 -- 172.21.15.112:0/166518126 shutdown_connections 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 -- 172.21.15.112:0/166518126 wait complete. 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 Processor -- start 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.062+0000 7ff011eda640 1 -- start start 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff011eda640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c1006a0 0x7ff00c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff011eda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 0x7ff00c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff011eda640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c102250 0x7ff00c19f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff011eda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d5c0 con 0x7ff00c101890 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff011eda640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d440 con 0x7ff00c102250 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff011eda640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d740 con 0x7ff00c1006a0 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 0x7ff00c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 0x7ff00c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34768/0 (socket says 172.21.15.112:34768) 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 -- 172.21.15.112:0/1056428429 learned_addr learned my addr 172.21.15.112:0/1056428429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00bfff640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c102250 0x7ff00c19f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 -- 172.21.15.112:0/1056428429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c1006a0 msgr2=0x7ff00c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00b7fe640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c1006a0 0x7ff00c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c1006a0 0x7ff00c197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 -- 172.21.15.112:0/1056428429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c102250 msgr2=0x7ff00c19f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c102250 0x7ff00c19f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 -- 172.21.15.112:0/1056428429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c19f860 con 0x7ff00c101890 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00bfff640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c102250 0x7ff00c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff00affd640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 0x7ff00c198190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff000009a20 tx=0x7ff000004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff008ff9640 1 -- 172.21.15.112:0/1056428429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00003f030 con 0x7ff00c101890 2024-04-24T23:12:41.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c19faf0 con 0x7ff00c101890 2024-04-24T23:12:41.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff008ff9640 1 -- 172.21.15.112:0/1056428429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff000003d00 con 0x7ff00c101890 2024-04-24T23:12:41.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff008ff9640 1 -- 172.21.15.112:0/1056428429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000035070 con 0x7ff00c101890 2024-04-24T23:12:41.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.063+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c1a0030 con 0x7ff00c101890 2024-04-24T23:12:41.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.064+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c1079c0 con 0x7ff00c101890 2024-04-24T23:12:41.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.065+0000 7ff008ff9640 1 -- 172.21.15.112:0/1056428429 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff000042030 con 0x7ff00c101890 2024-04-24T23:12:41.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.066+0000 7ff008ff9640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.066+0000 7ff00b7fe640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.066+0000 7ff008ff9640 1 -- 172.21.15.112:0/1056428429 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff0000efdb0 con 0x7ff00c101890 2024-04-24T23:12:41.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.067+0000 7ff00b7fe640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff00c199120 tx=0x7feffc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.070+0000 7ff008ff9640 1 -- 172.21.15.112:0/1056428429 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff0000bcad0 con 0x7ff00c101890 2024-04-24T23:12:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:40 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4233934631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:40 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/671578378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:41.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:40 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2355934895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:41.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.245+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff00c02d330 con 0x7ff00c101890 2024-04-24T23:12:41.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.245+0000 7ff008ff9640 1 -- 172.21.15.112:0/1056428429 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7ff0000bc3d0 con 0x7ff00c101890 2024-04-24T23:12:41.247 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:12:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 msgr2=0x7feff40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff00c199120 tx=0x7feffc005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:41.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 msgr2=0x7ff00c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 0x7ff00c198190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff000009a20 tx=0x7ff000004270 comp rx=0 tx=0).stop 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 shutdown_connections 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c102250 0x7ff00c19f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c101890 0x7ff00c198190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 --2- 172.21.15.112:0/1056428429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c1006a0 0x7ff00c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 >> 172.21.15.112:0/1056428429 conn(0x7ff00c0fbc10 msgr2=0x7ff00c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 shutdown_connections 2024-04-24T23:12:41.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.247+0000 7ff011eda640 1 -- 172.21.15.112:0/1056428429 wait complete. 2024-04-24T23:12:41.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:12:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:40 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4233934631' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:40 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/671578378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:40 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2355934895' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.357+0000 7f4f40e9e640 1 -- 172.21.15.112:0/41625977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 msgr2=0x7f4f3c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.357+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/41625977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 0x7f4f3c100aa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009ca0 tx=0x7f4f2c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.357+0000 7f4f40e9e640 1 -- 172.21.15.112:0/41625977 shutdown_connections 2024-04-24T23:12:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.357+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/41625977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f3c102230 0x7f4f3c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.357+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/41625977 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f3c101870 0x7f4f3c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.357+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/41625977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 0x7f4f3c100aa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.357+0000 7f4f40e9e640 1 -- 172.21.15.112:0/41625977 >> 172.21.15.112:0/41625977 conn(0x7f4f3c0fbbf0 msgr2=0x7f4f3c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.357+0000 7f4f40e9e640 1 -- 172.21.15.112:0/41625977 shutdown_connections 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 -- 172.21.15.112:0/41625977 wait complete. 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 Processor -- start 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 -- start start 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 0x7f4f3c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f3c101870 0x7f4f3c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f3c102230 0x7f4f3c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c10c430 con 0x7f4f3c100680 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c10c2b0 con 0x7f4f3c101870 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f40e9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c10c5b0 con 0x7f4f3c102230 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f3a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 0x7f4f3c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f3a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 0x7f4f3c069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34788/0 (socket says 172.21.15.112:34788) 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.358+0000 7f4f3a59c640 1 -- 172.21.15.112:0/4165901082 learned_addr learned my addr 172.21.15.112:0/4165901082 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:41.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f39d9b640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f3c101870 0x7f4f3c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f3ad9d640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f3c102230 0x7f4f3c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f3a59c640 1 -- 172.21.15.112:0/4165901082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f3c102230 msgr2=0x7f4f3c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f3a59c640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f3c102230 0x7f4f3c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f3a59c640 1 -- 172.21.15.112:0/4165901082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f3c101870 msgr2=0x7f4f3c19c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f3a59c640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f3c101870 0x7f4f3c19c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f3a59c640 1 -- 172.21.15.112:0/4165901082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c19f820 con 0x7f4f3c100680 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f3ad9d640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f3c102230 0x7f4f3c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f3a59c640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 0x7f4f3c069440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c000f80 tx=0x7f4f2c031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f2b7fe640 1 -- 172.21.15.112:0/4165901082 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c02fb60 con 0x7f4f3c100680 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f2b7fe640 1 -- 172.21.15.112:0/4165901082 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4f2c02fd00 con 0x7f4f3c100680 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c19fa50 con 0x7f4f3c100680 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f2b7fe640 1 -- 172.21.15.112:0/4165901082 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c046910 con 0x7f4f3c100680 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.359+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c19ffa0 con 0x7f4f3c100680 2024-04-24T23:12:41.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.360+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3c1079a0 con 0x7f4f3c100680 2024-04-24T23:12:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.361+0000 7f4f2b7fe640 1 -- 172.21.15.112:0/4165901082 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4f2c030c30 con 0x7f4f3c100680 2024-04-24T23:12:41.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.362+0000 7f4f2b7fe640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f240a2b90 0x7f4f240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.362+0000 7f4f39d9b640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f240a2b90 0x7f4f240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.362+0000 7f4f2b7fe640 1 -- 172.21.15.112:0/4165901082 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4f2c0f0760 con 0x7f4f3c100680 2024-04-24T23:12:41.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.362+0000 7f4f39d9b640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f240a2b90 0x7f4f240a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f300095f0 tx=0x7f4f30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.366+0000 7f4f2b7fe640 1 -- 172.21.15.112:0/4165901082 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4f2c0bd480 con 0x7f4f3c100680 2024-04-24T23:12:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.555+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4f3c062340 con 0x7f4f3c100680 2024-04-24T23:12:41.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.555+0000 7f4f2b7fe640 1 -- 172.21.15.112:0/4165901082 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+601 (secure 0 0 0) 0x7f4f2c0bcd80 con 0x7f4f3c100680 2024-04-24T23:12:41.557 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1532442772 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2220790491 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/342211680 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2319978079 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2957535572 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1556121872 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/3348191830 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/3348191830 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4091217333 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:41.558 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2319978079 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-04-24T23:12:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f240a2b90 msgr2=0x7f4f240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f240a2b90 0x7f4f240a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f300095f0 tx=0x7f4f30005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:41.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 msgr2=0x7f4f3c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 0x7f4f3c069440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c000f80 tx=0x7f4f2c031ec0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 shutdown_connections 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f4f240a2b90 0x7f4f240a5050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f3c102230 0x7f4f3c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4f3c101870 0x7f4f3c19c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 --2- 172.21.15.112:0/4165901082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f3c100680 0x7f4f3c069440 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 >> 172.21.15.112:0/4165901082 conn(0x7f4f3c0fbbf0 msgr2=0x7f4f3c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 shutdown_connections 2024-04-24T23:12:41.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.557+0000 7f4f40e9e640 1 -- 172.21.15.112:0/4165901082 wait complete. 2024-04-24T23:12:41.570 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:12:41.570 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:12:41.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 -- 172.21.15.112:0/4077141253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60101890 msgr2=0x7f6a60101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 --2- 172.21.15.112:0/4077141253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60101890 0x7f6a60101d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009900 tx=0x7f6a4802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 -- 172.21.15.112:0/4077141253 shutdown_connections 2024-04-24T23:12:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 --2- 172.21.15.112:0/4077141253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a60102250 0x7f6a60108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 --2- 172.21.15.112:0/4077141253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a60101890 0x7f6a60101d10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 --2- 172.21.15.112:0/4077141253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a601006a0 0x7f6a60100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 -- 172.21.15.112:0/4077141253 >> 172.21.15.112:0/4077141253 conn(0x7f6a600fbc30 msgr2=0x7f6a600fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 -- 172.21.15.112:0/4077141253 shutdown_connections 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.723+0000 7f6a654a3640 1 -- 172.21.15.112:0/4077141253 wait complete. 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 Processor -- start 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 -- start start 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a601006a0 0x7f6a60197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60101890 0x7f6a601980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a60102250 0x7f6a6019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010d560 con 0x7f6a601006a0 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010d3e0 con 0x7f6a60102250 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6010d6e0 con 0x7f6a60101890 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a601006a0 0x7f6a60197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a601006a0 0x7f6a60197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34812/0 (socket says 172.21.15.112:34812) 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 -- 172.21.15.112:0/3905175883 learned_addr learned my addr 172.21.15.112:0/3905175883 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a57fff640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60101890 0x7f6a601980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5f7fe640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a60102250 0x7f6a6019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 -- 172.21.15.112:0/3905175883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60101890 msgr2=0x7f6a601980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60101890 0x7f6a601980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 -- 172.21.15.112:0/3905175883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a60102250 msgr2=0x7f6a6019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a60102250 0x7f6a6019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 -- 172.21.15.112:0/3905175883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6019f7d0 con 0x7f6a601006a0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5effd640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a601006a0 0x7f6a60197bb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000d950 tx=0x7f6a5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5cff9640 1 -- 172.21.15.112:0/3905175883 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50017020 con 0x7f6a601006a0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5cff9640 1 -- 172.21.15.112:0/3905175883 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6a50009c30 con 0x7f6a601006a0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6019fac0 con 0x7f6a601006a0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.724+0000 7f6a5cff9640 1 -- 172.21.15.112:0/3905175883 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000f7b0 con 0x7f6a601006a0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.725+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a601a0030 con 0x7f6a601006a0 2024-04-24T23:12:41.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.725+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a601079c0 con 0x7f6a601006a0 2024-04-24T23:12:41.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.726+0000 7f6a5cff9640 1 -- 172.21.15.112:0/3905175883 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6a5001a020 con 0x7f6a601006a0 2024-04-24T23:12:41.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.727+0000 7f6a5cff9640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:41.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.727+0000 7f6a57fff640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.727+0000 7f6a5cff9640 1 -- 172.21.15.112:0/3905175883 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6a500cf140 con 0x7f6a601006a0 2024-04-24T23:12:41.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.727+0000 7f6a57fff640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006010 tx=0x7f6a48031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:41.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.731+0000 7f6a5cff9640 1 -- 172.21.15.112:0/3905175883 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6a50021020 con 0x7f6a601006a0 2024-04-24T23:12:41.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.911+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a600623e0 con 0x7f6a601006a0 2024-04-24T23:12:41.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.913+0000 7f6a5cff9640 1 -- 172.21.15.112:0/3905175883 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7f6a50009de0 con 0x7f6a601006a0 2024-04-24T23:12:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:41.915 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a4c0a2c10 msgr2=0x7f6a4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6a48006010 tx=0x7f6a48031640 comp rx=0 tx=0).stop 2024-04-24T23:12:41.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a601006a0 msgr2=0x7f6a60197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a601006a0 0x7f6a60197bb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000d950 tx=0x7f6a5000de10 comp rx=0 tx=0).stop 2024-04-24T23:12:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 shutdown_connections 2024-04-24T23:12:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6a4c0a2c10 0x7f6a4c0a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a60102250 0x7f6a6019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60101890 0x7f6a601980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 --2- 172.21.15.112:0/3905175883 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a601006a0 0x7f6a60197bb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:41.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 >> 172.21.15.112:0/3905175883 conn(0x7f6a600fbc30 msgr2=0x7f6a60104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:41.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 shutdown_connections 2024-04-24T23:12:41.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:41.915+0000 7f6a654a3640 1 -- 172.21.15.112:0/3905175883 wait complete. 2024-04-24T23:12:41.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:12:41.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:12:42.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 -- 172.21.15.112:0/4134239110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4102980 msgr2=0x7fb1d4102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 --2- 172.21.15.112:0/4134239110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4102980 0x7fb1d4102e00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009900 tx=0x7fb1c402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 -- 172.21.15.112:0/4134239110 shutdown_connections 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 --2- 172.21.15.112:0/4134239110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d4103340 0x7fb1d4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 --2- 172.21.15.112:0/4134239110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4102980 0x7fb1d4102e00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 --2- 172.21.15.112:0/4134239110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4101790 0x7fb1d4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 -- 172.21.15.112:0/4134239110 >> 172.21.15.112:0/4134239110 conn(0x7fb1d40fcd20 msgr2=0x7fb1d40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 -- 172.21.15.112:0/4134239110 shutdown_connections 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.028+0000 7fb1da763640 1 -- 172.21.15.112:0/4134239110 wait complete. 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1da763640 1 Processor -- start 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1da763640 1 -- start start 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1da763640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4101790 0x7fb1d4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1da763640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d4102980 0x7fb1d41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1da763640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4103340 0x7fb1d419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1da763640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d410d540 con 0x7fb1d4103340 2024-04-24T23:12:42.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1da763640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d410d3c0 con 0x7fb1d4101790 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1da763640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d410d6c0 con 0x7fb1d4102980 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d8d00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4103340 0x7fb1d419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d8d00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4103340 0x7fb1d419f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34838/0 (socket says 172.21.15.112:34838) 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d8d00640 1 -- 172.21.15.112:0/1547124814 learned_addr learned my addr 172.21.15.112:0/1547124814 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d37fe640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d4102980 0x7fb1d41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d3fff640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4101790 0x7fb1d4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d8d00640 1 -- 172.21.15.112:0/1547124814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d4102980 msgr2=0x7fb1d41980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d8d00640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d4102980 0x7fb1d41980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d8d00640 1 -- 172.21.15.112:0/1547124814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4101790 msgr2=0x7fb1d4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d8d00640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4101790 0x7fb1d4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d8d00640 1 -- 172.21.15.112:0/1547124814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d419f780 con 0x7fb1d4103340 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.029+0000 7fb1d3fff640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4101790 0x7fb1d4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:42.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.030+0000 7fb1d8d00640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4103340 0x7fb1d419f110 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00eb10 tx=0x7fb1cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.030+0000 7fb1d17fa640 1 -- 172.21.15.112:0/1547124814 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc004b20 con 0x7fb1d4103340 2024-04-24T23:12:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.030+0000 7fb1d17fa640 1 -- 172.21.15.112:0/1547124814 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb1cc004cc0 con 0x7fb1d4103340 2024-04-24T23:12:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.030+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d419fa70 con 0x7fb1d4103340 2024-04-24T23:12:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.030+0000 7fb1d17fa640 1 -- 172.21.15.112:0/1547124814 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0107f0 con 0x7fb1d4103340 2024-04-24T23:12:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.030+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d419ff90 con 0x7fb1d4103340 2024-04-24T23:12:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.031+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d4108ab0 con 0x7fb1d4103340 2024-04-24T23:12:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.032+0000 7fb1d17fa640 1 -- 172.21.15.112:0/1547124814 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb1cc005020 con 0x7fb1d4103340 2024-04-24T23:12:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.032+0000 7fb1d17fa640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1bc0a2b90 0x7fb1bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.032+0000 7fb1d3fff640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1bc0a2b90 0x7fb1bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.032+0000 7fb1d17fa640 1 -- 172.21.15.112:0/1547124814 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb1cc0cf870 con 0x7fb1d4103340 2024-04-24T23:12:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.033+0000 7fb1d3fff640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1bc0a2b90 0x7fb1bc0a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb1d41027e0 tx=0x7fb1c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.037+0000 7fb1d17fa640 1 -- 172.21.15.112:0/1547124814 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb1cc09c590 con 0x7fb1d4103340 2024-04-24T23:12:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.211+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1d4062340 con 0x7fb1d4103340 2024-04-24T23:12:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.211+0000 7fb1d17fa640 1 -- 172.21.15.112:0/1547124814 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fb1cc09be90 con 0x7fb1d4103340 2024-04-24T23:12:42.213 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:12:42.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1bc0a2b90 msgr2=0x7fb1bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1bc0a2b90 0x7fb1bc0a5050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb1d41027e0 tx=0x7fb1c8005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:42.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4103340 msgr2=0x7fb1d419f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4103340 0x7fb1d419f110 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00eb10 tx=0x7fb1cc00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 shutdown_connections 2024-04-24T23:12:42.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb1bc0a2b90 0x7fb1bc0a5050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1d4103340 0x7fb1d419f110 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1d4102980 0x7fb1d41980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 --2- 172.21.15.112:0/1547124814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1d4101790 0x7fb1d4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 >> 172.21.15.112:0/1547124814 conn(0x7fb1d40fcd20 msgr2=0x7fb1d4105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 shutdown_connections 2024-04-24T23:12:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.213+0000 7fb1da763640 1 -- 172.21.15.112:0/1547124814 wait complete. 2024-04-24T23:12:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:41 smithi112 ceph-mon[29186]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 116 B/s wr, 0 op/s 2024-04-24T23:12:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3720966757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1056428429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.225 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4165901082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:12:42.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:12:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:41 smithi187 ceph-mon[33292]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 116 B/s wr, 0 op/s 2024-04-24T23:12:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3720966757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1056428429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4165901082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:12:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:41 smithi150 ceph-mon[32247]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 116 B/s wr, 0 op/s 2024-04-24T23:12:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3720966757' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1056428429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4165901082' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 -- 172.21.15.112:0/2729241489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 msgr2=0x7f9634103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 --2- 172.21.15.112:0/2729241489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 0x7f9634103190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9624009900 tx=0x7f962402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 -- 172.21.15.112:0/2729241489 shutdown_connections 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 --2- 172.21.15.112:0/2729241489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96341001a0 0x7f9634109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 --2- 172.21.15.112:0/2729241489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96340ff7e0 0x7f96340ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 --2- 172.21.15.112:0/2729241489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 0x7f9634103190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 -- 172.21.15.112:0/2729241489 >> 172.21.15.112:0/2729241489 conn(0x7f96340fb380 msgr2=0x7f96340fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 -- 172.21.15.112:0/2729241489 shutdown_connections 2024-04-24T23:12:42.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 -- 172.21.15.112:0/2729241489 wait complete. 2024-04-24T23:12:42.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.326+0000 7f9639142640 1 Processor -- start 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f9639142640 1 -- start start 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f9639142640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96340ff7e0 0x7f9634102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f9639142640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96341001a0 0x7f9634100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f9639142640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 0x7f9634100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f9639142640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f963410d570 con 0x7f9634102d70 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f9639142640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f963410d3f0 con 0x7f96341001a0 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f9639142640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f963410d6f0 con 0x7f96340ff7e0 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 0x7f9634100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 0x7f9634100ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34848/0 (socket says 172.21.15.112:34848) 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 -- 172.21.15.112:0/2384276590 learned_addr learned my addr 172.21.15.112:0/2384276590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f9632d9d640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96340ff7e0 0x7f9634102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963259c640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96341001a0 0x7f9634100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 -- 172.21.15.112:0/2384276590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96340ff7e0 msgr2=0x7f9634102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96340ff7e0 0x7f9634102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 -- 172.21.15.112:0/2384276590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96341001a0 msgr2=0x7f9634100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96341001a0 0x7f9634100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 -- 172.21.15.112:0/2384276590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9634101880 con 0x7f9634102d70 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.327+0000 7f963359e640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 0x7f9634100ff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f962c00cce0 tx=0x7f962c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.328+0000 7f961bfff640 1 -- 172.21.15.112:0/2384276590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00f450 con 0x7f9634102d70 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.328+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96341a1c50 con 0x7f9634102d70 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.328+0000 7f961bfff640 1 -- 172.21.15.112:0/2384276590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f962c00faf0 con 0x7f9634102d70 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.328+0000 7f961bfff640 1 -- 172.21.15.112:0/2384276590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c018cc0 con 0x7f9634102d70 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.328+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96341a2150 con 0x7f9634102d70 2024-04-24T23:12:42.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.329+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9634108bd0 con 0x7f9634102d70 2024-04-24T23:12:42.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.330+0000 7f961bfff640 1 -- 172.21.15.112:0/2384276590 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f962c00f5f0 con 0x7f9634102d70 2024-04-24T23:12:42.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.330+0000 7f961bfff640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f961c0a2b90 0x7f961c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.331+0000 7f9632d9d640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f961c0a2b90 0x7f961c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.331+0000 7f961bfff640 1 -- 172.21.15.112:0/2384276590 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f962c0cf6e0 con 0x7f9634102d70 2024-04-24T23:12:42.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.331+0000 7f9632d9d640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f961c0a2b90 0x7f961c0a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9624031570 tx=0x7f9624031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.335+0000 7f961bfff640 1 -- 172.21.15.112:0/2384276590 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f962c014030 con 0x7f9634102d70 2024-04-24T23:12:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.510+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96340623e0 con 0x7f9634102d70 2024-04-24T23:12:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.511+0000 7f961bfff640 1 -- 172.21.15.112:0/2384276590 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f962c09c400 con 0x7f9634102d70 2024-04-24T23:12:42.512 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.512+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f961c0a2b90 msgr2=0x7f961c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f961c0a2b90 0x7f961c0a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9624031570 tx=0x7f9624031640 comp rx=0 tx=0).stop 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 msgr2=0x7f9634100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 0x7f9634100ff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f962c00cce0 tx=0x7f962c00a620 comp rx=0 tx=0).stop 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 shutdown_connections 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f961c0a2b90 0x7f961c0a5050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9634102d70 0x7f9634100ff0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96341001a0 0x7f9634100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 --2- 172.21.15.112:0/2384276590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96340ff7e0 0x7f9634102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 >> 172.21.15.112:0/2384276590 conn(0x7f96340fb380 msgr2=0x7f9634105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 shutdown_connections 2024-04-24T23:12:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.513+0000 7f9639142640 1 -- 172.21.15.112:0/2384276590 wait complete. 2024-04-24T23:12:42.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:12:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.625+0000 7f082f522640 1 -- 172.21.15.112:0/1271967060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 msgr2=0x7f0828101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.625+0000 7f082f522640 1 --2- 172.21.15.112:0/1271967060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 0x7f0828101bc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0818009900 tx=0x7f081802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 -- 172.21.15.112:0/1271967060 shutdown_connections 2024-04-24T23:12:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 --2- 172.21.15.112:0/1271967060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0828103350 0x7f0828109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 --2- 172.21.15.112:0/1271967060 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0828102990 0x7f0828102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 --2- 172.21.15.112:0/1271967060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 0x7f0828101bc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 -- 172.21.15.112:0/1271967060 >> 172.21.15.112:0/1271967060 conn(0x7f08280fcd10 msgr2=0x7f08280ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 -- 172.21.15.112:0/1271967060 shutdown_connections 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 -- 172.21.15.112:0/1271967060 wait complete. 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 Processor -- start 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 -- start start 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 0x7f0828197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0828102990 0x7f08281980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0828103350 0x7f082819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f082810d540 con 0x7f08281017a0 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f082810d3c0 con 0x7f0828103350 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082f522640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f082810d6c0 con 0x7f0828102990 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082d2be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 0x7f0828197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082d2be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 0x7f0828197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34856/0 (socket says 172.21.15.112:34856) 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082d2be640 1 -- 172.21.15.112:0/1844021303 learned_addr learned my addr 172.21.15.112:0/1844021303 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.626+0000 7f082cabd640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0828102990 0x7f08281980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082dabf640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0828103350 0x7f082819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082d2be640 1 -- 172.21.15.112:0/1844021303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0828102990 msgr2=0x7f08281980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082d2be640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0828102990 0x7f08281980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082d2be640 1 -- 172.21.15.112:0/1844021303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0828103350 msgr2=0x7f082819f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082d2be640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0828103350 0x7f082819f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082d2be640 1 -- 172.21.15.112:0/1844021303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082819f7a0 con 0x7f08281017a0 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082dabf640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0828103350 0x7f082819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082d2be640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 0x7f0828197b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0818005ab0 tx=0x7f08180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f081e7fc640 1 -- 172.21.15.112:0/1844021303 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081803f030 con 0x7f08281017a0 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f081e7fc640 1 -- 172.21.15.112:0/1844021303 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0818003d00 con 0x7f08281017a0 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082819fa30 con 0x7f08281017a0 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f081e7fc640 1 -- 172.21.15.112:0/1844021303 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818035070 con 0x7f08281017a0 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.627+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082819ff20 con 0x7f08281017a0 2024-04-24T23:12:42.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.628+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0828108ac0 con 0x7f08281017a0 2024-04-24T23:12:42.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.629+0000 7f081e7fc640 1 -- 172.21.15.112:0/1844021303 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0818042030 con 0x7f08281017a0 2024-04-24T23:12:42.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.630+0000 7f081e7fc640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08140a2b90 0x7f08140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.630+0000 7f082cabd640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08140a2b90 0x7f08140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.630+0000 7f081e7fc640 1 -- 172.21.15.112:0/1844021303 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f08180ef8e0 con 0x7f08281017a0 2024-04-24T23:12:42.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.630+0000 7f082cabd640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08140a2b90 0x7f08140a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0828199090 tx=0x7f082400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.634+0000 7f081e7fc640 1 -- 172.21.15.112:0/1844021303 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08180bc600 con 0x7f08281017a0 2024-04-24T23:12:42.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.811+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0828062340 con 0x7f08281017a0 2024-04-24T23:12:42.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.811+0000 7f081e7fc640 1 -- 172.21.15.112:0/1844021303 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f08180bbf00 con 0x7f08281017a0 2024-04-24T23:12:42.813 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:12:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08140a2b90 msgr2=0x7f08140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08140a2b90 0x7f08140a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0828199090 tx=0x7f082400b410 comp rx=0 tx=0).stop 2024-04-24T23:12:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 msgr2=0x7f0828197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 0x7f0828197b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0818005ab0 tx=0x7f08180042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 shutdown_connections 2024-04-24T23:12:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08140a2b90 0x7f08140a5050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0828103350 0x7f082819f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0828102990 0x7f08281980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 --2- 172.21.15.112:0/1844021303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08281017a0 0x7f0828197b80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 >> 172.21.15.112:0/1844021303 conn(0x7f08280fcd10 msgr2=0x7f0828105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 shutdown_connections 2024-04-24T23:12:42.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.813+0000 7f082f522640 1 -- 172.21.15.112:0/1844021303 wait complete. 2024-04-24T23:12:42.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 -- 172.21.15.112:0/888944832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284102dc0 msgr2=0x7f32841031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 --2- 172.21.15.112:0/888944832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284102dc0 0x7f32841031e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f327c009ca0 tx=0x7f327c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 -- 172.21.15.112:0/888944832 shutdown_connections 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 --2- 172.21.15.112:0/888944832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32841001f0 0x7f3284109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 --2- 172.21.15.112:0/888944832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32840ff830 0x7f32840ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 --2- 172.21.15.112:0/888944832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284102dc0 0x7f32841031e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 -- 172.21.15.112:0/888944832 >> 172.21.15.112:0/888944832 conn(0x7f32840fb370 msgr2=0x7f32840fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 -- 172.21.15.112:0/888944832 shutdown_connections 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 -- 172.21.15.112:0/888944832 wait complete. 2024-04-24T23:12:42.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 Processor -- start 2024-04-24T23:12:42.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.925+0000 7f3289015640 1 -- start start 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3289015640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32840ff830 0x7f3284197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3289015640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841001f0 0x7f3284198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3289015640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3284102dc0 0x7f328419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3289015640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f328410d580 con 0x7f32841001f0 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3289015640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f328410d400 con 0x7f32840ff830 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3289015640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f328410d700 con 0x7f3284102dc0 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841001f0 0x7f3284198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841001f0 0x7f3284198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34870/0 (socket says 172.21.15.112:34870) 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 -- 172.21.15.112:0/3095306407 learned_addr learned my addr 172.21.15.112:0/3095306407 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328359e640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3284102dc0 0x7f328419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3282d9d640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32840ff830 0x7f3284197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 -- 172.21.15.112:0/3095306407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3284102dc0 msgr2=0x7f328419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3284102dc0 0x7f328419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 -- 172.21.15.112:0/3095306407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32840ff830 msgr2=0x7f3284197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32840ff830 0x7f3284197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 -- 172.21.15.112:0/3095306407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328419f860 con 0x7f32841001f0 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f328259c640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841001f0 0x7f3284198190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f326c00d950 tx=0x7f326c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f327bfff640 1 -- 172.21.15.112:0/3095306407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c017070 con 0x7f32841001f0 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f327bfff640 1 -- 172.21.15.112:0/3095306407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f326c009c30 con 0x7f32841001f0 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328419fb50 con 0x7f32841001f0 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f327bfff640 1 -- 172.21.15.112:0/3095306407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c00f740 con 0x7f32841001f0 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.926+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32841a00a0 con 0x7f32841001f0 2024-04-24T23:12:42.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.928+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3284108c20 con 0x7f32841001f0 2024-04-24T23:12:42.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.928+0000 7f327bfff640 1 -- 172.21.15.112:0/3095306407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f326c010a80 con 0x7f32841001f0 2024-04-24T23:12:42.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.929+0000 7f327bfff640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32700a2b90 0x7f32700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:42.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.929+0000 7f3282d9d640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32700a2b90 0x7f32700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:42.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.929+0000 7f327bfff640 1 -- 172.21.15.112:0/3095306407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f326c0cef60 con 0x7f32841001f0 2024-04-24T23:12:42.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.929+0000 7f3282d9d640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32700a2b90 0x7f32700a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f327c006010 tx=0x7f327c0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:42.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:42.933+0000 7f327bfff640 1 -- 172.21.15.112:0/3095306407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f326c009de0 con 0x7f32841001f0 2024-04-24T23:12:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.113+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3284062340 con 0x7f32841001f0 2024-04-24T23:12:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.113+0000 7f327bfff640 1 -- 172.21.15.112:0/3095306407 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f326c09bc80 con 0x7f32841001f0 2024-04-24T23:12:43.116 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32700a2b90 msgr2=0x7f32700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32700a2b90 0x7f32700a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f327c006010 tx=0x7f327c0093f0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841001f0 msgr2=0x7f3284198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841001f0 0x7f3284198190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f326c00d950 tx=0x7f326c00de10 comp rx=0 tx=0).stop 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 shutdown_connections 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32700a2b90 0x7f32700a5050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3284102dc0 0x7f328419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841001f0 0x7f3284198190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 --2- 172.21.15.112:0/3095306407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32840ff830 0x7f3284197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 >> 172.21.15.112:0/3095306407 conn(0x7f32840fb370 msgr2=0x7f3284105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 shutdown_connections 2024-04-24T23:12:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.116+0000 7f3289015640 1 -- 172.21.15.112:0/3095306407 wait complete. 2024-04-24T23:12:43.127 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3905175883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:43.127 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1547124814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.128 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2384276590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.128 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1844021303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1210427353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 msgr2=0x7ff054100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1210427353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 0x7ff054100e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff04002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1210427353 shutdown_connections 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1210427353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff054101350 0x7ff0541017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1210427353 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 0x7ff054100e10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1210427353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff054100050 0x7ff054100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1210427353 >> 172.21.15.112:0/1210427353 conn(0x7ff0540fbbf0 msgr2=0x7ff0540fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1210427353 shutdown_connections 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.226+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1210427353 wait complete. 2024-04-24T23:12:43.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff05ad1d640 1 Processor -- start 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff05ad1d640 1 -- start start 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff05ad1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff054100050 0x7ff054199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff05ad1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 0x7ff05419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff05ad1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff054101350 0x7ff0541a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff05ad1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff05410f740 con 0x7ff0541009b0 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff05ad1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff05410f5c0 con 0x7ff054100050 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff05ad1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff05410f8c0 con 0x7ff054101350 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff053fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 0x7ff05419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.227+0000 7ff053fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 0x7ff05419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34890/0 (socket says 172.21.15.112:34890) 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff053fff640 1 -- 172.21.15.112:0/1076303780 learned_addr learned my addr 172.21.15.112:0/1076303780 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff058ab9640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff054100050 0x7ff054199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff0592ba640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff054101350 0x7ff0541a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff053fff640 1 -- 172.21.15.112:0/1076303780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff054101350 msgr2=0x7ff0541a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff053fff640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff054101350 0x7ff0541a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff053fff640 1 -- 172.21.15.112:0/1076303780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff054100050 msgr2=0x7ff054199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff053fff640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff054100050 0x7ff054199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff053fff640 1 -- 172.21.15.112:0/1076303780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0541a1990 con 0x7ff0541009b0 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff058ab9640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff054100050 0x7ff054199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff053fff640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 0x7ff05419a2b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff040009a20 tx=0x7ff040004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff051ffb640 1 -- 172.21.15.112:0/1076303780 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04003f030 con 0x7ff0541009b0 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff051ffb640 1 -- 172.21.15.112:0/1076303780 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff040003d00 con 0x7ff0541009b0 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff051ffb640 1 -- 172.21.15.112:0/1076303780 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040035070 con 0x7ff0541009b0 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0541a1c20 con 0x7ff0541009b0 2024-04-24T23:12:43.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.228+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0541a20c0 con 0x7ff0541009b0 2024-04-24T23:12:43.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.230+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05410aed0 con 0x7ff0541009b0 2024-04-24T23:12:43.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.230+0000 7ff051ffb640 1 -- 172.21.15.112:0/1076303780 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff040042030 con 0x7ff0541009b0 2024-04-24T23:12:43.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.231+0000 7ff051ffb640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0440a2c10 0x7ff0440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:43.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.231+0000 7ff058ab9640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0440a2c10 0x7ff0440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:43.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.231+0000 7ff051ffb640 1 -- 172.21.15.112:0/1076303780 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff0400ef980 con 0x7ff0541009b0 2024-04-24T23:12:43.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.231+0000 7ff058ab9640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0440a2c10 0x7ff0440a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff048005d00 tx=0x7ff048005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:43.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3905175883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:43.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1547124814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2384276590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1844021303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.235+0000 7ff051ffb640 1 -- 172.21.15.112:0/1076303780 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff0400bc6a0 con 0x7ff0541009b0 2024-04-24T23:12:43.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3905175883' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:43.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1547124814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2384276590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1844021303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:12:43.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.422+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff054062340 con 0x7ff0541009b0 2024-04-24T23:12:43.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.423+0000 7ff051ffb640 1 -- 172.21.15.112:0/1076303780 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+601 (secure 0 0 0) 0x7ff0400bbfa0 con 0x7ff0541009b0 2024-04-24T23:12:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1532442772 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2220790491 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/342211680 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2319978079 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2957535572 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1556121872 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/3348191830 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:43.426 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/3348191830 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:43.426 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4091217333 2024-04-25T23:05:00.376626+0000 2024-04-24T23:12:43.426 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2319978079 2024-04-25T23:05:26.702079+0000 2024-04-24T23:12:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0440a2c10 msgr2=0x7ff0440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0440a2c10 0x7ff0440a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff048005d00 tx=0x7ff048005c70 comp rx=0 tx=0).stop 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 msgr2=0x7ff05419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 0x7ff05419a2b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff040009a20 tx=0x7ff040004430 comp rx=0 tx=0).stop 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 shutdown_connections 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0440a2c10 0x7ff0440a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff054101350 0x7ff0541a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0541009b0 0x7ff05419a2b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 --2- 172.21.15.112:0/1076303780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff054100050 0x7ff054199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:43.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 >> 172.21.15.112:0/1076303780 conn(0x7ff0540fbbf0 msgr2=0x7ff0540fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:43.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 shutdown_connections 2024-04-24T23:12:43.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:43.425+0000 7ff05ad1d640 1 -- 172.21.15.112:0/1076303780 wait complete. 2024-04-24T23:12:43.438 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2024-04-24T23:12:43.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:43.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-24T23:12:43.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi112.stderr:ceph-fuse[73878]: fuse finished with error 0 and tester_r 0 2024-04-24T23:12:43.462 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:12:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:43 smithi187 ceph-mon[33292]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-04-24T23:12:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:43 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3095306407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:43 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1076303780' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:12:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:43 smithi112 ceph-mon[29186]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-04-24T23:12:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:43 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3095306407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:43 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1076303780' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:12:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:43 smithi150 ceph-mon[32247]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 0 op/s 2024-04-24T23:12:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:43 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3095306407' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:12:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:43 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1076303780' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:12:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:45 smithi187 ceph-mon[33292]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 443 B/s wr, 0 op/s 2024-04-24T23:12:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:45 smithi112 ceph-mon[29186]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 443 B/s wr, 0 op/s 2024-04-24T23:12:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:45 smithi150 ceph-mon[32247]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 443 B/s wr, 0 op/s 2024-04-24T23:12:48.119 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:47 smithi112 ceph-mon[29186]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:47 smithi187 ceph-mon[33292]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:48.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:47 smithi150 ceph-mon[32247]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:49.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-24T23:12:49.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:12:49.564 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-24T23:12:49.587 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:49.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.724+0000 7fe98bddc640 1 -- 172.21.15.112:0/2518538457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984100680 msgr2=0x7fe984100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.724+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2518538457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984100680 0x7fe984100aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe980009ca0 tx=0x7fe98002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.724+0000 7fe98bddc640 1 -- 172.21.15.112:0/2518538457 shutdown_connections 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.724+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2518538457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe984102230 0x7fe984108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.724+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2518538457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe984101870 0x7fe984101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.724+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2518538457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984100680 0x7fe984100aa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.724+0000 7fe98bddc640 1 -- 172.21.15.112:0/2518538457 >> 172.21.15.112:0/2518538457 conn(0x7fe9840fbbf0 msgr2=0x7fe9840fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 -- 172.21.15.112:0/2518538457 shutdown_connections 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 -- 172.21.15.112:0/2518538457 wait complete. 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 Processor -- start 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 -- start start 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe984100680 0x7fe984197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984101870 0x7fe984198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:49.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe984102230 0x7fe98419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe98410c4e0 con 0x7fe984101870 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe98410c360 con 0x7fe984100680 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe98bddc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe98410c660 con 0x7fe984102230 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe989377640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984101870 0x7fe984198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe989377640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984101870 0x7fe984198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34908/0 (socket says 172.21.15.112:34908) 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe989377640 1 -- 172.21.15.112:0/2040986282 learned_addr learned my addr 172.21.15.112:0/2040986282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.725+0000 7fe989b78640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe984100680 0x7fe984197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe98a379640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe984102230 0x7fe98419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe989377640 1 -- 172.21.15.112:0/2040986282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe984102230 msgr2=0x7fe98419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe989377640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe984102230 0x7fe98419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe989377640 1 -- 172.21.15.112:0/2040986282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe984100680 msgr2=0x7fe984197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe989377640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe984100680 0x7fe984197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe989377640 1 -- 172.21.15.112:0/2040986282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98419f800 con 0x7fe984101870 2024-04-24T23:12:49.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe989377640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984101870 0x7fe984198120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe97400b4a0 tx=0x7fe97400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:49.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe97affd640 1 -- 172.21.15.112:0/2040986282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974014070 con 0x7fe984101870 2024-04-24T23:12:49.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98419faf0 con 0x7fe984101870 2024-04-24T23:12:49.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe97affd640 1 -- 172.21.15.112:0/2040986282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe974010b10 con 0x7fe984101870 2024-04-24T23:12:49.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe97affd640 1 -- 172.21.15.112:0/2040986282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974017d60 con 0x7fe984101870 2024-04-24T23:12:49.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.726+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9841a0040 con 0x7fe984101870 2024-04-24T23:12:49.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.727+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9841a02d0 con 0x7fe984101870 2024-04-24T23:12:49.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.728+0000 7fe97affd640 1 -- 172.21.15.112:0/2040986282 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe974017070 con 0x7fe984101870 2024-04-24T23:12:49.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.729+0000 7fe97affd640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe9700a2b90 0x7fe9700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:49.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.729+0000 7fe989b78640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe9700a2b90 0x7fe9700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:49.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.729+0000 7fe97affd640 1 -- 172.21.15.112:0/2040986282 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe9740ce5a0 con 0x7fe984101870 2024-04-24T23:12:49.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.729+0000 7fe989b78640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe9700a2b90 0x7fe9700a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe980000f80 tx=0x7fe980005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:49.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.734+0000 7fe97affd640 1 -- 172.21.15.112:0/2040986282 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe97409ac70 con 0x7fe984101870 2024-04-24T23:12:49.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.913+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9841a04a0 con 0x7fe984101870 2024-04-24T23:12:49.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.914+0000 7fe97affd640 1 -- 172.21.15.112:0/2040986282 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7fe97409ef70 con 0x7fe984101870 2024-04-24T23:12:49.916 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:49.917 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe9700a2b90 msgr2=0x7fe9700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe9700a2b90 0x7fe9700a5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe980000f80 tx=0x7fe980005ef0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984101870 msgr2=0x7fe984198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:49.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984101870 0x7fe984198120 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe97400b4a0 tx=0x7fe97400b960 comp rx=0 tx=0).stop 2024-04-24T23:12:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 shutdown_connections 2024-04-24T23:12:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe9700a2b90 0x7fe9700a5050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe984102230 0x7fe98419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe984101870 0x7fe984198120 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 --2- 172.21.15.112:0/2040986282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe984100680 0x7fe984197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 >> 172.21.15.112:0/2040986282 conn(0x7fe9840fbbf0 msgr2=0x7fe9840fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 shutdown_connections 2024-04-24T23:12:49.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:49.916+0000 7fe98bddc640 1 -- 172.21.15.112:0/2040986282 wait complete. 2024-04-24T23:12:49.927 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:49 smithi112 ceph-mon[29186]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:49.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 -- 172.21.15.112:0/242669919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910100690 msgr2=0x7fe910100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 --2- 172.21.15.112:0/242669919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910100690 0x7fe910100ab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009900 tx=0x7fe90c02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 -- 172.21.15.112:0/242669919 shutdown_connections 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 --2- 172.21.15.112:0/242669919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe910102240 0x7fe910108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 --2- 172.21.15.112:0/242669919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe910101880 0x7fe910101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 --2- 172.21.15.112:0/242669919 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910100690 0x7fe910100ab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 -- 172.21.15.112:0/242669919 >> 172.21.15.112:0/242669919 conn(0x7fe9100fbc20 msgr2=0x7fe9100fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 -- 172.21.15.112:0/242669919 shutdown_connections 2024-04-24T23:12:50.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.025+0000 7fe918361640 1 -- 172.21.15.112:0/242669919 wait complete. 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe918361640 1 Processor -- start 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe918361640 1 -- start start 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe918361640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe910100690 0x7fe91006a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe918361640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910101880 0x7fe91019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe918361640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe910102240 0x7fe91019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe918361640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010c440 con 0x7fe910101880 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe918361640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010c2c0 con 0x7fe910102240 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe918361640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe91010c5c0 con 0x7fe910100690 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910101880 0x7fe91019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910101880 0x7fe91019c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34924/0 (socket says 172.21.15.112:34924) 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 -- 172.21.15.112:0/3816044941 learned_addr learned my addr 172.21.15.112:0/3816044941 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:50.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9160fd640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe910100690 0x7fe91006a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 -- 172.21.15.112:0/3816044941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe910100690 msgr2=0x7fe91006a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe910100690 0x7fe91006a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 -- 172.21.15.112:0/3816044941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe910102240 msgr2=0x7fe91019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe910102240 0x7fe91019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 -- 172.21.15.112:0/3816044941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91019f7f0 con 0x7fe910101880 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9160fd640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe910100690 0x7fe91006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.026+0000 7fe9158fc640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910101880 0x7fe91019c8f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c890 tx=0x7fe90000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.027+0000 7fe9077fe640 1 -- 172.21.15.112:0/3816044941 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900004220 con 0x7fe910101880 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.027+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91019fae0 con 0x7fe910101880 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.027+0000 7fe9077fe640 1 -- 172.21.15.112:0/3816044941 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe9000043c0 con 0x7fe910101880 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.027+0000 7fe9077fe640 1 -- 172.21.15.112:0/3816044941 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900004b10 con 0x7fe910101880 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.027+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9101a0030 con 0x7fe910101880 2024-04-24T23:12:50.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.028+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9101079b0 con 0x7fe910101880 2024-04-24T23:12:50.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.029+0000 7fe9077fe640 1 -- 172.21.15.112:0/3816044941 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe9000051b0 con 0x7fe910101880 2024-04-24T23:12:50.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.030+0000 7fe9077fe640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe8fc0a2b90 0x7fe8fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.030+0000 7fe9160fd640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe8fc0a2b90 0x7fe8fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.030+0000 7fe9077fe640 1 -- 172.21.15.112:0/3816044941 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe9000cf370 con 0x7fe910101880 2024-04-24T23:12:50.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.030+0000 7fe9160fd640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe8fc0a2b90 0x7fe8fc0a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe90c000c00 tx=0x7fe90c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.033+0000 7fe9077fe640 1 -- 172.21.15.112:0/3816044941 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe900014030 con 0x7fe910101880 2024-04-24T23:12:50.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.210+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe91002d3c0 con 0x7fe910101880 2024-04-24T23:12:50.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.211+0000 7fe9077fe640 1 -- 172.21.15.112:0/3816044941 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7fe90009c090 con 0x7fe910101880 2024-04-24T23:12:50.214 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:50.214 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe8fc0a2b90 msgr2=0x7fe8fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe8fc0a2b90 0x7fe8fc0a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe90c000c00 tx=0x7fe90c009560 comp rx=0 tx=0).stop 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910101880 msgr2=0x7fe91019c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910101880 0x7fe91019c8f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c890 tx=0x7fe90000cd50 comp rx=0 tx=0).stop 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 shutdown_connections 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe8fc0a2b90 0x7fe8fc0a5050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe910102240 0x7fe91019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe910101880 0x7fe91019c8f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 --2- 172.21.15.112:0/3816044941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe910100690 0x7fe91006a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.213+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 >> 172.21.15.112:0/3816044941 conn(0x7fe9100fbc20 msgr2=0x7fe910104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.214+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 shutdown_connections 2024-04-24T23:12:50.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.214+0000 7fe918361640 1 -- 172.21.15.112:0/3816044941 wait complete. 2024-04-24T23:12:50.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-24T23:12:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:49 smithi187 ceph-mon[33292]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:49 smithi150 ceph-mon[32247]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2395724697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b0102250 msgr2=0x7ff0b0108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2395724697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b0102250 0x7ff0b0108ae0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009900 tx=0x7ff0a802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2395724697 shutdown_connections 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2395724697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b0102250 0x7ff0b0108ae0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2395724697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0101890 0x7ff0b0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2395724697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0b01006a0 0x7ff0b0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2395724697 >> 172.21.15.112:0/2395724697 conn(0x7ff0b00fbc30 msgr2=0x7ff0b00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2395724697 shutdown_connections 2024-04-24T23:12:50.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.325+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2395724697 wait complete. 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0b4bd5640 1 Processor -- start 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0b4bd5640 1 -- start start 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0b4bd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b01006a0 0x7ff0b0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0b4bd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0101890 0x7ff0b0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0b4bd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0b0102250 0x7ff0b019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0b4bd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b010d550 con 0x7ff0b01006a0 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0b4bd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b010d3d0 con 0x7ff0b0101890 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0b4bd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b010d6d0 con 0x7ff0b0102250 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b01006a0 0x7ff0b0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b01006a0 0x7ff0b0197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34944/0 (socket says 172.21.15.112:34944) 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 -- 172.21.15.112:0/2696433759 learned_addr learned my addr 172.21.15.112:0/2696433759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0aed9d640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0b0102250 0x7ff0b019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0add9b640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0101890 0x7ff0b0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 -- 172.21.15.112:0/2696433759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0b0102250 msgr2=0x7ff0b019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0b0102250 0x7ff0b019f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 -- 172.21.15.112:0/2696433759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0101890 msgr2=0x7ff0b0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0101890 0x7ff0b0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 -- 172.21.15.112:0/2696433759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b019f760 con 0x7ff0b01006a0 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0aed9d640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0b0102250 0x7ff0b019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.326+0000 7ff0ae59c640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b01006a0 0x7ff0b0197b40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000eaa0 tx=0x7ff0a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.327+0000 7ff09f7fe640 1 -- 172.21.15.112:0/2696433759 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0016070 con 0x7ff0b01006a0 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.327+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b019fa50 con 0x7ff0b01006a0 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.327+0000 7ff09f7fe640 1 -- 172.21.15.112:0/2696433759 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0a000ce10 con 0x7ff0b01006a0 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.327+0000 7ff09f7fe640 1 -- 172.21.15.112:0/2696433759 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00116d0 con 0x7ff0b01006a0 2024-04-24T23:12:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.327+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b019ff50 con 0x7ff0b01006a0 2024-04-24T23:12:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.328+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b0107960 con 0x7ff0b01006a0 2024-04-24T23:12:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.329+0000 7ff09f7fe640 1 -- 172.21.15.112:0/2696433759 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff0a0019070 con 0x7ff0b01006a0 2024-04-24T23:12:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.329+0000 7ff09f7fe640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0980a2b90 0x7ff0980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.329+0000 7ff0add9b640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0980a2b90 0x7ff0980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.329+0000 7ff09f7fe640 1 -- 172.21.15.112:0/2696433759 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff0a00cf440 con 0x7ff0b01006a0 2024-04-24T23:12:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.330+0000 7ff0add9b640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0980a2b90 0x7ff0980a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0199050 tx=0x7ff0a400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.334+0000 7ff09f7fe640 1 -- 172.21.15.112:0/2696433759 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff0a009c160 con 0x7ff0b01006a0 2024-04-24T23:12:50.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.547+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff0b0062370 con 0x7ff0b01006a0 2024-04-24T23:12:50.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.870+0000 7ff09f7fe640 1 -- 172.21.15.112:0/2696433759 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7ff0a009ba60 con 0x7ff0b01006a0 2024-04-24T23:12:50.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0980a2b90 msgr2=0x7ff0980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0980a2b90 0x7ff0980a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0199050 tx=0x7ff0a400a4c0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b01006a0 msgr2=0x7ff0b0197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b01006a0 0x7ff0b0197b40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000eaa0 tx=0x7ff0a000ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 shutdown_connections 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0980a2b90 0x7ff0980a5050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0b0102250 0x7ff0b019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0b0101890 0x7ff0b0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 --2- 172.21.15.112:0/2696433759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0b01006a0 0x7ff0b0197b40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 >> 172.21.15.112:0/2696433759 conn(0x7ff0b00fbc30 msgr2=0x7ff0b0104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 shutdown_connections 2024-04-24T23:12:50.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.872+0000 7ff0b4bd5640 1 -- 172.21.15.112:0/2696433759 wait complete. 2024-04-24T23:12:50.885 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-24T23:12:50.885 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-24T23:12:50.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 -- 172.21.15.112:0/66966448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 msgr2=0x7f5a40109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 --2- 172.21.15.112:0/66966448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 0x7f5a40109bd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009ca0 tx=0x7f5a3802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 -- 172.21.15.112:0/66966448 shutdown_connections 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 --2- 172.21.15.112:0/66966448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 0x7f5a40109bd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 --2- 172.21.15.112:0/66966448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40102980 0x7f5a40102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 --2- 172.21.15.112:0/66966448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a40101790 0x7f5a40101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 -- 172.21.15.112:0/66966448 >> 172.21.15.112:0/66966448 conn(0x7f5a400fcd20 msgr2=0x7f5a400ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 -- 172.21.15.112:0/66966448 shutdown_connections 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.984+0000 7f5a46142640 1 -- 172.21.15.112:0/66966448 wait complete. 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a46142640 1 Processor -- start 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a46142640 1 -- start start 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a46142640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40101790 0x7f5a40197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a46142640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a40102980 0x7f5a401980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a46142640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 0x7f5a4019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a46142640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010d540 con 0x7f5a40103340 2024-04-24T23:12:50.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a46142640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010d3c0 con 0x7f5a40102980 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a46142640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4010d6c0 con 0x7f5a40101790 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 0x7f5a4019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 0x7f5a4019f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37816/0 (socket says 172.21.15.112:37816) 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 -- 172.21.15.112:0/1539421603 learned_addr learned my addr 172.21.15.112:0/1539421603 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3effd640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a40102980 0x7f5a401980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3f7fe640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40101790 0x7f5a40197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 -- 172.21.15.112:0/1539421603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40101790 msgr2=0x7f5a40197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40101790 0x7f5a40197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 -- 172.21.15.112:0/1539421603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a40102980 msgr2=0x7f5a401980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a40102980 0x7f5a401980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 -- 172.21.15.112:0/1539421603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4019f7c0 con 0x7f5a40103340 2024-04-24T23:12:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.985+0000 7f5a3ffff640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 0x7f5a4019f150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005e30 tx=0x7f5a38031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.986+0000 7f5a3cff9640 1 -- 172.21.15.112:0/1539421603 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3802fb60 con 0x7f5a40103340 2024-04-24T23:12:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.986+0000 7f5a3cff9640 1 -- 172.21.15.112:0/1539421603 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5a3802fd00 con 0x7f5a40103340 2024-04-24T23:12:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.986+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4019fa50 con 0x7f5a40103340 2024-04-24T23:12:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.986+0000 7f5a3cff9640 1 -- 172.21.15.112:0/1539421603 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38046980 con 0x7f5a40103340 2024-04-24T23:12:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.986+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4019ffa0 con 0x7f5a40103340 2024-04-24T23:12:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.987+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a40108a50 con 0x7f5a40103340 2024-04-24T23:12:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.988+0000 7f5a3cff9640 1 -- 172.21.15.112:0/1539421603 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5a38030be0 con 0x7f5a40103340 2024-04-24T23:12:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.988+0000 7f5a3cff9640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a280a2b90 0x7f5a280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.988+0000 7f5a3f7fe640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a280a2b90 0x7f5a280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.988+0000 7f5a3cff9640 1 -- 172.21.15.112:0/1539421603 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5a380f0800 con 0x7f5a40103340 2024-04-24T23:12:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.989+0000 7f5a3f7fe640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a280a2b90 0x7f5a280a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005fd0 tx=0x7f5a30005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:50.993+0000 7f5a3cff9640 1 -- 172.21.15.112:0/1539421603 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5a380bd520 con 0x7f5a40103340 2024-04-24T23:12:51.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.213+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f5a40062340 con 0x7f5a40103340 2024-04-24T23:12:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:50 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2040986282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:50 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3816044941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:50 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2696433759' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:12:51.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2040986282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3816044941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2696433759' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:12:51.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:50 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2040986282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:50 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3816044941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:51.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:50 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2696433759' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-24T23:12:51.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.872+0000 7f5a3cff9640 1 -- 172.21.15.112:0/1539421603 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f5a380bce20 con 0x7f5a40103340 2024-04-24T23:12:51.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a280a2b90 msgr2=0x7f5a280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a280a2b90 0x7f5a280a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005fd0 tx=0x7f5a30005850 comp rx=0 tx=0).stop 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 msgr2=0x7f5a4019f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 0x7f5a4019f150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005e30 tx=0x7f5a38031ee0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 shutdown_connections 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f5a280a2b90 0x7f5a280a5050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a40103340 0x7f5a4019f150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a40102980 0x7f5a401980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 --2- 172.21.15.112:0/1539421603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40101790 0x7f5a40197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 >> 172.21.15.112:0/1539421603 conn(0x7f5a400fcd20 msgr2=0x7f5a40105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 shutdown_connections 2024-04-24T23:12:51.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.874+0000 7f5a46142640 1 -- 172.21.15.112:0/1539421603 wait complete. 2024-04-24T23:12:51.891 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-24T23:12:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- 172.21.15.112:0/2274282169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364102980 msgr2=0x7f3364102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 --2- 172.21.15.112:0/2274282169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364102980 0x7f3364102e00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3354009900 tx=0x7f335402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- 172.21.15.112:0/2274282169 shutdown_connections 2024-04-24T23:12:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 --2- 172.21.15.112:0/2274282169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3364103340 0x7f3364109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 --2- 172.21.15.112:0/2274282169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364102980 0x7f3364102e00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 --2- 172.21.15.112:0/2274282169 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364101790 0x7f3364101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- 172.21.15.112:0/2274282169 >> 172.21.15.112:0/2274282169 conn(0x7f33640fcd20 msgr2=0x7f33640ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:51.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- 172.21.15.112:0/2274282169 shutdown_connections 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- 172.21.15.112:0/2274282169 wait complete. 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 Processor -- start 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- start start 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364101790 0x7f3364197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364102980 0x7f3364198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3364103340 0x7f336419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f336410d540 con 0x7f3364101790 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f336410d3c0 con 0x7f3364102980 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.992+0000 7f336bfd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f336410d6c0 con 0x7f3364103340 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364101790 0x7f3364197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364101790 0x7f3364197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37832/0 (socket says 172.21.15.112:37832) 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 -- 172.21.15.112:0/4179462294 learned_addr learned my addr 172.21.15.112:0/4179462294 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f336a572640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3364103340 0x7f336419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369570640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364102980 0x7f3364198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 -- 172.21.15.112:0/4179462294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3364103340 msgr2=0x7f336419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3364103340 0x7f336419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 -- 172.21.15.112:0/4179462294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364102980 msgr2=0x7f3364198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364102980 0x7f3364198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 -- 172.21.15.112:0/4179462294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336419f770 con 0x7f3364101790 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369570640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364102980 0x7f3364198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f3369d71640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364101790 0x7f3364197b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f336000c970 tx=0x7f336000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f335affd640 1 -- 172.21.15.112:0/4179462294 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360004220 con 0x7f3364101790 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f335affd640 1 -- 172.21.15.112:0/4179462294 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f33600043c0 con 0x7f3364101790 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f335affd640 1 -- 172.21.15.112:0/4179462294 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336000f6b0 con 0x7f3364101790 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336419fa60 con 0x7f3364101790 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.993+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336419ff80 con 0x7f3364101790 2024-04-24T23:12:51.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.994+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3364108ab0 con 0x7f3364101790 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.995+0000 7f335affd640 1 -- 172.21.15.112:0/4179462294 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f33600037a0 con 0x7f3364101790 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.996+0000 7f335affd640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f33500a2b90 0x7f33500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.996+0000 7f3369570640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f33500a2b90 0x7f33500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.996+0000 7f335affd640 1 -- 172.21.15.112:0/4179462294 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f33600cefc0 con 0x7f3364101790 2024-04-24T23:12:51.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:51.996+0000 7f3369570640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f33500a2b90 0x7f33500a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f3354031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:52.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.000+0000 7f335affd640 1 -- 172.21.15.112:0/4179462294 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3360014030 con 0x7f3364101790 2024-04-24T23:12:52.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.217+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f3364062340 con 0x7f3364101790 2024-04-24T23:12:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:51 smithi187 ceph-mon[33292]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:51 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2696433759' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:12:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:51 smithi187 ceph-mon[33292]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:51 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1539421603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T23:12:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:51 smithi112 ceph-mon[29186]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:51 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2696433759' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:12:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:51 smithi112 ceph-mon[29186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:51 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1539421603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T23:12:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:51 smithi150 ceph-mon[32247]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:51 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2696433759' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-24T23:12:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:51 smithi150 ceph-mon[32247]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:51 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1539421603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-24T23:12:52.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.882+0000 7f335affd640 1 -- 172.21.15.112:0/4179462294 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f3360003e40 con 0x7f3364101790 2024-04-24T23:12:52.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f33500a2b90 msgr2=0x7f33500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f33500a2b90 0x7f33500a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3354000c00 tx=0x7f3354031640 comp rx=0 tx=0).stop 2024-04-24T23:12:52.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364101790 msgr2=0x7f3364197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:52.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364101790 0x7f3364197b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f336000c970 tx=0x7f336000ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:52.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 shutdown_connections 2024-04-24T23:12:52.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f33500a2b90 0x7f33500a5050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3364103340 0x7f336419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3364102980 0x7f3364198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 --2- 172.21.15.112:0/4179462294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3364101790 0x7f3364197b50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:52.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 >> 172.21.15.112:0/4179462294 conn(0x7f33640fcd20 msgr2=0x7f3364105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:52.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 shutdown_connections 2024-04-24T23:12:52.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:52.885+0000 7f336bfd5640 1 -- 172.21.15.112:0/4179462294 wait complete. 2024-04-24T23:12:52.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-24T23:12:53.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 -- 172.21.15.112:0/3757575937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 msgr2=0x7f73ec101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3757575937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 0x7f73ec101cf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009900 tx=0x7f73dc02a410 comp rx=0 tx=0).stop 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 -- 172.21.15.112:0/3757575937 shutdown_connections 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3757575937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ec102230 0x7f73ec108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3757575937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 0x7f73ec101cf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3757575937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73ec100680 0x7f73ec100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 -- 172.21.15.112:0/3757575937 >> 172.21.15.112:0/3757575937 conn(0x7f73ec0fbbf0 msgr2=0x7f73ec0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 -- 172.21.15.112:0/3757575937 shutdown_connections 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.001+0000 7f73f46d2640 1 -- 172.21.15.112:0/3757575937 wait complete. 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f46d2640 1 Processor -- start 2024-04-24T23:12:53.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f46d2640 1 -- start start 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f46d2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73ec100680 0x7f73ec069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f46d2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 0x7f73ec19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f46d2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ec102230 0x7f73ec19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f46d2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10c430 con 0x7f73ec101870 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f46d2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10c2b0 con 0x7f73ec102230 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f46d2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec10c5b0 con 0x7f73ec100680 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f1c6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 0x7f73ec19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f1c6d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 0x7f73ec19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37840/0 (socket says 172.21.15.112:37840) 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f1c6d640 1 -- 172.21.15.112:0/3182409733 learned_addr learned my addr 172.21.15.112:0/3182409733 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f2c6f640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ec102230 0x7f73ec19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f1c6d640 1 -- 172.21.15.112:0/3182409733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73ec100680 msgr2=0x7f73ec069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f1c6d640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73ec100680 0x7f73ec069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f1c6d640 1 -- 172.21.15.112:0/3182409733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ec102230 msgr2=0x7f73ec19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f1c6d640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ec102230 0x7f73ec19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f1c6d640 1 -- 172.21.15.112:0/3182409733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec19f820 con 0x7f73ec101870 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.002+0000 7f73f2c6f640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ec102230 0x7f73ec19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.003+0000 7f73f1c6d640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 0x7f73ec19c8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009a20 tx=0x7f73dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.003+0000 7f73e37fe640 1 -- 172.21.15.112:0/3182409733 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc03f030 con 0x7f73ec101870 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.003+0000 7f73e37fe640 1 -- 172.21.15.112:0/3182409733 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f73dc003d00 con 0x7f73ec101870 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.003+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec19fa50 con 0x7f73ec101870 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.003+0000 7f73e37fe640 1 -- 172.21.15.112:0/3182409733 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc035070 con 0x7f73ec101870 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.003+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec19ff40 con 0x7f73ec101870 2024-04-24T23:12:53.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.004+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec1079c0 con 0x7f73ec101870 2024-04-24T23:12:53.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.005+0000 7f73e37fe640 1 -- 172.21.15.112:0/3182409733 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f73dc042030 con 0x7f73ec101870 2024-04-24T23:12:53.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.005+0000 7f73e37fe640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73d80a2b90 0x7f73d80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.005+0000 7f73f246e640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73d80a2b90 0x7f73d80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.005+0000 7f73e37fe640 1 -- 172.21.15.112:0/3182409733 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f73dc0efa00 con 0x7f73ec101870 2024-04-24T23:12:53.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.006+0000 7f73f246e640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73d80a2b90 0x7f73d80a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f73ec1016d0 tx=0x7f73e8005c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.010+0000 7f73e37fe640 1 -- 172.21.15.112:0/3182409733 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73dc0bc720 con 0x7f73ec101870 2024-04-24T23:12:53.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.220+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f73ec062370 con 0x7f73ec101870 2024-04-24T23:12:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:52 smithi187 ceph-mon[33292]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:12:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:52 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1539421603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T23:12:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:52 smithi187 ceph-mon[33292]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:12:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:52 smithi187 ceph-mon[33292]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:52 smithi187 ceph-mon[33292]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T23:12:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:52 smithi187 ceph-mon[33292]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T23:12:53.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:52 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4179462294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:12:53.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:52 smithi112 ceph-mon[29186]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:12:53.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:52 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1539421603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T23:12:53.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:52 smithi112 ceph-mon[29186]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:12:53.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:52 smithi112 ceph-mon[29186]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:53.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:52 smithi112 ceph-mon[29186]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T23:12:53.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:52 smithi112 ceph-mon[29186]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T23:12:53.322 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:52 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4179462294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:12:53.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:52 smithi150 ceph-mon[32247]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:12:53.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:52 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1539421603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-24T23:12:53.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:52 smithi150 ceph-mon[32247]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-24T23:12:53.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:52 smithi150 ceph-mon[32247]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-24T23:12:53.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:52 smithi150 ceph-mon[32247]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-24T23:12:53.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:52 smithi150 ceph-mon[32247]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-24T23:12:53.323 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:52 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4179462294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-24T23:12:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.858+0000 7f73e37fe640 1 -- 172.21.15.112:0/3182409733 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f73dc0bc020 con 0x7f73ec101870 2024-04-24T23:12:53.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73d80a2b90 msgr2=0x7f73d80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73d80a2b90 0x7f73d80a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f73ec1016d0 tx=0x7f73e8005c10 comp rx=0 tx=0).stop 2024-04-24T23:12:53.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 msgr2=0x7f73ec19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 0x7f73ec19c8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009a20 tx=0x7f73dc004270 comp rx=0 tx=0).stop 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 shutdown_connections 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73d80a2b90 0x7f73d80a5050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73ec102230 0x7f73ec19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73ec101870 0x7f73ec19c8f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 --2- 172.21.15.112:0/3182409733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73ec100680 0x7f73ec069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.860+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 >> 172.21.15.112:0/3182409733 conn(0x7f73ec0fbbf0 msgr2=0x7f73ec104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.861+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 shutdown_connections 2024-04-24T23:12:53.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.861+0000 7f73f46d2640 1 -- 172.21.15.112:0/3182409733 wait complete. 2024-04-24T23:12:53.879 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-24T23:12:53.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-24T23:12:53.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 -- 172.21.15.112:0/221515299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c1074c0 msgr2=0x7f097c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 --2- 172.21.15.112:0/221515299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c1074c0 0x7f097c1098d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0974009ca0 tx=0x7f097402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 -- 172.21.15.112:0/221515299 shutdown_connections 2024-04-24T23:12:53.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 --2- 172.21.15.112:0/221515299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c1074c0 0x7f097c1098d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 --2- 172.21.15.112:0/221515299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f097c104b90 0x7f097c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 --2- 172.21.15.112:0/221515299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f097c102260 0x7f097c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 -- 172.21.15.112:0/221515299 >> 172.21.15.112:0/221515299 conn(0x7f097c0fbc30 msgr2=0x7f097c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:53.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 -- 172.21.15.112:0/221515299 shutdown_connections 2024-04-24T23:12:53.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.985+0000 7f0983807640 1 -- 172.21.15.112:0/221515299 wait complete. 2024-04-24T23:12:53.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 Processor -- start 2024-04-24T23:12:53.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 -- start start 2024-04-24T23:12:53.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f097c102260 0x7f097c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c104b90 0x7f097c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f097c1074c0 0x7f097c19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f097c10d570 con 0x7f097c104b90 2024-04-24T23:12:53.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f097c10d3f0 con 0x7f097c102260 2024-04-24T23:12:53.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f097c10d6f0 con 0x7f097c1074c0 2024-04-24T23:12:53.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c104b90 0x7f097c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c104b90 0x7f097c198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37862/0 (socket says 172.21.15.112:37862) 2024-04-24T23:12:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 -- 172.21.15.112:0/2498360979 learned_addr learned my addr 172.21.15.112:0/2498360979 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f09815a3640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f097c102260 0x7f097c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0981da4640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f097c1074c0 0x7f097c19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 -- 172.21.15.112:0/2498360979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f097c1074c0 msgr2=0x7f097c19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f097c1074c0 0x7f097c19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 -- 172.21.15.112:0/2498360979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f097c102260 msgr2=0x7f097c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f097c102260 0x7f097c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 -- 172.21.15.112:0/2498360979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c19f790 con 0x7f097c104b90 2024-04-24T23:12:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0980da2640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c104b90 0x7f097c198100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f096c00c970 tx=0x7f096c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f09727fc640 1 -- 172.21.15.112:0/2498360979 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c007bb0 con 0x7f097c104b90 2024-04-24T23:12:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f09727fc640 1 -- 172.21.15.112:0/2498360979 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f096c007d50 con 0x7f097c104b90 2024-04-24T23:12:53.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f09727fc640 1 -- 172.21.15.112:0/2498360979 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c018860 con 0x7f097c104b90 2024-04-24T23:12:53.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.986+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c19fa80 con 0x7f097c104b90 2024-04-24T23:12:53.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.987+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c19ff80 con 0x7f097c104b90 2024-04-24T23:12:53.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.987+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097c0fd720 con 0x7f097c104b90 2024-04-24T23:12:53.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.989+0000 7f09727fc640 1 -- 172.21.15.112:0/2498360979 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f096c018a80 con 0x7f097c104b90 2024-04-24T23:12:53.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.989+0000 7f09727fc640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f09680a2b90 0x7f09680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:53.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.989+0000 7f09815a3640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f09680a2b90 0x7f09680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:53.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.990+0000 7f09727fc640 1 -- 172.21.15.112:0/2498360979 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f096c0cf300 con 0x7f097c104b90 2024-04-24T23:12:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.990+0000 7f09815a3640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f09680a2b90 0x7f09680a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0978009bd0 tx=0x7f0978005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:53.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:53.994+0000 7f09727fc640 1 -- 172.21.15.112:0/2498360979 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f096c014030 con 0x7f097c104b90 2024-04-24T23:12:54.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.196+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f097c061b80 con 0x7f09680a2b90 2024-04-24T23:12:54.204 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:54.204 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4179462294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:12:54.204 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] up:active 2024-04-24T23:12:54.204 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:12:54.204 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T23:12:54.204 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-24T23:12:54.204 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-24T23:12:54.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3182409733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:12:54.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3182409733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:12:54.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] up:active 2024-04-24T23:12:54.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T23:12:54.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:12:54.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-24T23:12:54.205 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:53 smithi112 ceph-mon[29186]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-24T23:12:54.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.219+0000 7f09727fc640 1 -- 172.21.15.112:0/2498360979 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f097c061b80 con 0x7f09680a2b90 2024-04-24T23:12:54.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f09680a2b90 msgr2=0x7f09680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f09680a2b90 0x7f09680a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0978009bd0 tx=0x7f0978005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:54.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c104b90 msgr2=0x7f097c198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c104b90 0x7f097c198100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f096c00c970 tx=0x7f096c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 shutdown_connections 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f09680a2b90 0x7f09680a5050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f097c1074c0 0x7f097c19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f097c104b90 0x7f097c198100 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 --2- 172.21.15.112:0/2498360979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f097c102260 0x7f097c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 >> 172.21.15.112:0/2498360979 conn(0x7f097c0fbc30 msgr2=0x7f097c102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 shutdown_connections 2024-04-24T23:12:54.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.221+0000 7f0983807640 1 -- 172.21.15.112:0/2498360979 wait complete. 2024-04-24T23:12:54.234 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-24T23:12:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4179462294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] up:active 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3182409733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3182409733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] up:active 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T23:12:54.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:12:54.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-24T23:12:54.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:53 smithi187 ceph-mon[33292]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-24T23:12:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:12:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4179462294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-24T23:12:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] up:active 2024-04-24T23:12:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-24T23:12:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-24T23:12:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-24T23:12:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-24T23:12:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3182409733' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-24T23:12:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3182409733' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-24T23:12:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] up:active 2024-04-24T23:12:54.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-24T23:12:54.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-24T23:12:54.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-24T23:12:54.320 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:53 smithi150 ceph-mon[32247]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-24T23:12:54.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 -- 172.21.15.112:0/3252925049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f8104bb0 msgr2=0x7fb0f8106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3252925049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f8104bb0 0x7fb0f8106fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009900 tx=0x7fb0e002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 -- 172.21.15.112:0/3252925049 shutdown_connections 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3252925049 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0f81074e0 0x7fb0f81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3252925049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f8104bb0 0x7fb0f8106fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3252925049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0f8102280 0x7fb0f8104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 -- 172.21.15.112:0/3252925049 >> 172.21.15.112:0/3252925049 conn(0x7fb0f80fbc10 msgr2=0x7fb0f80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 -- 172.21.15.112:0/3252925049 shutdown_connections 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.335+0000 7fb0fce12640 1 -- 172.21.15.112:0/3252925049 wait complete. 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0fce12640 1 Processor -- start 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0fce12640 1 -- start start 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0fce12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0f8102280 0x7fb0f8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0fce12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0f8104bb0 0x7fb0f8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0fce12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81074e0 0x7fb0f819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0fce12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f810d550 con 0x7fb0f81074e0 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0fce12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f810d3d0 con 0x7fb0f8104bb0 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0fce12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f810d6d0 con 0x7fb0f8102280 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81074e0 0x7fb0f819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81074e0 0x7fb0f819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37904/0 (socket says 172.21.15.112:37904) 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 -- 172.21.15.112:0/3085294038 learned_addr learned my addr 172.21.15.112:0/3085294038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0effff640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0f8104bb0 0x7fb0f8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f659c640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0f8102280 0x7fb0f8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 -- 172.21.15.112:0/3085294038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0f8102280 msgr2=0x7fb0f8197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0f8102280 0x7fb0f8197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 -- 172.21.15.112:0/3085294038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0f8104bb0 msgr2=0x7fb0f8198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0f8104bb0 0x7fb0f8198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 -- 172.21.15.112:0/3085294038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f819f850 con 0x7fb0f81074e0 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f659c640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0f8102280 0x7fb0f8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:54.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0f6d9d640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81074e0 0x7fb0f819f1b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000eb10 tx=0x7fb0f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:54.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.336+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/3085294038 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0004b20 con 0x7fb0f81074e0 2024-04-24T23:12:54.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.337+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/3085294038 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb0f0004cc0 con 0x7fb0f81074e0 2024-04-24T23:12:54.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.337+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f819fb40 con 0x7fb0f81074e0 2024-04-24T23:12:54.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.337+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/3085294038 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00107f0 con 0x7fb0f81074e0 2024-04-24T23:12:54.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.337+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f81a0040 con 0x7fb0f81074e0 2024-04-24T23:12:54.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.338+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f80fd700 con 0x7fb0f81074e0 2024-04-24T23:12:54.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.339+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/3085294038 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb0f0005020 con 0x7fb0f81074e0 2024-04-24T23:12:54.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.339+0000 7fb0ef7fe640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0e40a2b90 0x7fb0e40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.339+0000 7fb0f659c640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0e40a2b90 0x7fb0e40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.339+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/3085294038 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb0f00cf960 con 0x7fb0f81074e0 2024-04-24T23:12:54.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.340+0000 7fb0f659c640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0e40a2b90 0x7fb0e40a5050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8199110 tx=0x7fb0e8005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:54.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.344+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/3085294038 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb0f009c680 con 0x7fb0f81074e0 2024-04-24T23:12:54.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:12:54 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:12:54.201+0000 7f63105a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:54.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:12:54 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:12:54.201+0000 7f63105a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:54.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:12:54 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:12:54.201+0000 7f63105a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:54.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:12:54 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:12:54.201+0000 7f63105a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:54.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:12:54 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:12:54.201+0000 7f63105a3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-24T23:12:54.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.547+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb0f8061b80 con 0x7fb0e40a2b90 2024-04-24T23:12:54.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.572+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/3085294038 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb0f8061b80 con 0x7fb0e40a2b90 2024-04-24T23:12:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0e40a2b90 msgr2=0x7fb0e40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0e40a2b90 0x7fb0e40a5050 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8199110 tx=0x7fb0e8005e70 comp rx=0 tx=0).stop 2024-04-24T23:12:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81074e0 msgr2=0x7fb0f819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81074e0 0x7fb0f819f1b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000eb10 tx=0x7fb0f000c2d0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 shutdown_connections 2024-04-24T23:12:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0e40a2b90 0x7fb0e40a5050 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0f81074e0 0x7fb0f819f1b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0f8104bb0 0x7fb0f8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 --2- 172.21.15.112:0/3085294038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0f8102280 0x7fb0f8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 >> 172.21.15.112:0/3085294038 conn(0x7fb0f80fbc10 msgr2=0x7fb0f8102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 shutdown_connections 2024-04-24T23:12:54.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.574+0000 7fb0fce12640 1 -- 172.21.15.112:0/3085294038 wait complete. 2024-04-24T23:12:54.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-24T23:12:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 -- 172.21.15.112:0/1078713616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40105330 msgr2=0x7fcc401057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1078713616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40105330 0x7fcc401057b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009ca0 tx=0x7fcc3802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 -- 172.21.15.112:0/1078713616 shutdown_connections 2024-04-24T23:12:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1078713616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40105330 0x7fcc401057b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1078713616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40102980 0x7fcc40104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1078713616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc40100010 0x7fcc40102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 -- 172.21.15.112:0/1078713616 >> 172.21.15.112:0/1078713616 conn(0x7fcc400fbc30 msgr2=0x7fcc400fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 -- 172.21.15.112:0/1078713616 shutdown_connections 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.687+0000 7fcc481f9640 1 -- 172.21.15.112:0/1078713616 wait complete. 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc481f9640 1 Processor -- start 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc481f9640 1 -- start start 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc481f9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40100010 0x7fcc4019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc481f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40102980 0x7fcc4019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc481f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc40105330 0x7fcc401a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc481f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40113b60 con 0x7fcc40102980 2024-04-24T23:12:54.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc481f9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401139e0 con 0x7fcc40100010 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc481f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40113ce0 con 0x7fcc40105330 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45794640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40102980 0x7fcc4019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45794640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40102980 0x7fcc4019e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37920/0 (socket says 172.21.15.112:37920) 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45794640 1 -- 172.21.15.112:0/1327668948 learned_addr learned my addr 172.21.15.112:0/1327668948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45f95640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40100010 0x7fcc4019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc46796640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc40105330 0x7fcc401a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45794640 1 -- 172.21.15.112:0/1327668948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc40105330 msgr2=0x7fcc401a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45794640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc40105330 0x7fcc401a56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45794640 1 -- 172.21.15.112:0/1327668948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40100010 msgr2=0x7fcc4019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45794640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40100010 0x7fcc4019e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.688+0000 7fcc45794640 1 -- 172.21.15.112:0/1327668948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc401a5d20 con 0x7fcc40102980 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.689+0000 7fcc45794640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40102980 0x7fcc4019e640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000ea60 tx=0x7fcc3000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.689+0000 7fcc36ffd640 1 -- 172.21.15.112:0/1327668948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000cde0 con 0x7fcc40102980 2024-04-24T23:12:54.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.689+0000 7fcc36ffd640 1 -- 172.21.15.112:0/1327668948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcc300035f0 con 0x7fcc40102980 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.689+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc401a5fb0 con 0x7fcc40102980 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.689+0000 7fcc36ffd640 1 -- 172.21.15.112:0/1327668948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30010720 con 0x7fcc40102980 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.689+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc401a64d0 con 0x7fcc40102980 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.690+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc400fd720 con 0x7fcc40102980 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.691+0000 7fcc36ffd640 1 -- 172.21.15.112:0/1327668948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcc300037a0 con 0x7fcc40102980 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.691+0000 7fcc36ffd640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc2c0a2b90 0x7fcc2c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.691+0000 7fcc45f95640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc2c0a2b90 0x7fcc2c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.691+0000 7fcc36ffd640 1 -- 172.21.15.112:0/1327668948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcc300cf5f0 con 0x7fcc40102980 2024-04-24T23:12:54.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.692+0000 7fcc45f95640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc2c0a2b90 0x7fcc2c0a5050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c005d00 tx=0x7fcc3c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:54.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.696+0000 7fcc36ffd640 1 -- 172.21.15.112:0/1327668948 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcc3009c310 con 0x7fcc40102980 2024-04-24T23:12:54.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.900+0000 7fcc36ffd640 1 -- 172.21.15.112:0/1327668948 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcc30098280 con 0x7fcc40102980 2024-04-24T23:12:54.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.910+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fcc40061b80 con 0x7fcc2c0a2b90 2024-04-24T23:12:54.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.915+0000 7fcc36ffd640 1 -- 172.21.15.112:0/1327668948 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fcc40061b80 con 0x7fcc2c0a2b90 2024-04-24T23:12:54.917 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_0/bc29c36b-b1c1-4efa-9a0c-12e1f62e2a96 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc2c0a2b90 msgr2=0x7fcc2c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc2c0a2b90 0x7fcc2c0a5050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c005d00 tx=0x7fcc3c00b410 comp rx=0 tx=0).stop 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40102980 msgr2=0x7fcc4019e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40102980 0x7fcc4019e640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000ea60 tx=0x7fcc3000ef20 comp rx=0 tx=0).stop 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 shutdown_connections 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcc2c0a2b90 0x7fcc2c0a5050 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc40105330 0x7fcc401a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc40102980 0x7fcc4019e640 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 --2- 172.21.15.112:0/1327668948 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc40100010 0x7fcc4019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:54.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.917+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 >> 172.21.15.112:0/1327668948 conn(0x7fcc400fbc30 msgr2=0x7fcc40103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:54.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.918+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 shutdown_connections 2024-04-24T23:12:54.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:54.918+0000 7fcc481f9640 1 -- 172.21.15.112:0/1327668948 wait complete. 2024-04-24T23:12:54.930 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-24T23:12:55.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 -- 172.21.15.112:0/508362735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 msgr2=0x7f6ed0109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/508362735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 0x7f6ed0109bd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009ca0 tx=0x7f6ec802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 -- 172.21.15.112:0/508362735 shutdown_connections 2024-04-24T23:12:55.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/508362735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 0x7f6ed0109bd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/508362735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed0102980 0x7f6ed0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/508362735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ed0101790 0x7f6ed0101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 -- 172.21.15.112:0/508362735 >> 172.21.15.112:0/508362735 conn(0x7f6ed00fcd20 msgr2=0x7f6ed00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 -- 172.21.15.112:0/508362735 shutdown_connections 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.032+0000 7f6ed7b90640 1 -- 172.21.15.112:0/508362735 wait complete. 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 Processor -- start 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 -- start start 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed0101790 0x7f6ed0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ed0102980 0x7f6ed01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 0x7f6ed019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010d540 con 0x7f6ed0103340 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010d3c0 con 0x7f6ed0102980 2024-04-24T23:12:55.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed010d6c0 con 0x7f6ed0101790 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 0x7f6ed019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 0x7f6ed019f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37932/0 (socket says 172.21.15.112:37932) 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 -- 172.21.15.112:0/3421260968 learned_addr learned my addr 172.21.15.112:0/3421260968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed512b640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ed0102980 0x7f6ed01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed592c640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed0101790 0x7f6ed0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 -- 172.21.15.112:0/3421260968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed0101790 msgr2=0x7f6ed0197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed0101790 0x7f6ed0197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 -- 172.21.15.112:0/3421260968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ed0102980 msgr2=0x7f6ed01980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ed0102980 0x7f6ed01980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 -- 172.21.15.112:0/3421260968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed019f7c0 con 0x7f6ed0103340 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed612d640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 0x7f6ed019f150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006010 tx=0x7f6ec8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ec6ffd640 1 -- 172.21.15.112:0/3421260968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec802fa20 con 0x7f6ed0103340 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed019fa50 con 0x7f6ed0103340 2024-04-24T23:12:55.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.033+0000 7f6ec6ffd640 1 -- 172.21.15.112:0/3421260968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6ec802fbc0 con 0x7f6ed0103340 2024-04-24T23:12:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.034+0000 7f6ec6ffd640 1 -- 172.21.15.112:0/3421260968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8046770 con 0x7f6ed0103340 2024-04-24T23:12:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.034+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed019ffa0 con 0x7f6ed0103340 2024-04-24T23:12:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.035+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed0108a50 con 0x7f6ed0103340 2024-04-24T23:12:55.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.036+0000 7f6ec6ffd640 1 -- 172.21.15.112:0/3421260968 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6ec8046910 con 0x7f6ed0103340 2024-04-24T23:12:55.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.036+0000 7f6ec6ffd640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6ebc0a2c20 0x7f6ebc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.037+0000 7f6ed592c640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6ebc0a2c20 0x7f6ebc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.037+0000 7f6ec6ffd640 1 -- 172.21.15.112:0/3421260968 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f6ec80f0650 con 0x7f6ed0103340 2024-04-24T23:12:55.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.037+0000 7f6ed592c640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6ebc0a2c20 0x7f6ebc0a50e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6ed01027e0 tx=0x7f6ec0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:55.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.041+0000 7f6ec6ffd640 1 -- 172.21.15.112:0/3421260968 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6ec80bd320 con 0x7f6ed0103340 2024-04-24T23:12:55.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.242+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6ed0061b80 con 0x7f6ebc0a2c20 2024-04-24T23:12:55.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.266+0000 7f6ec6ffd640 1 -- 172.21.15.112:0/3421260968 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6ed0061b80 con 0x7f6ebc0a2c20 2024-04-24T23:12:55.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6ebc0a2c20 msgr2=0x7f6ebc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6ebc0a2c20 0x7f6ebc0a50e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6ed01027e0 tx=0x7f6ec0005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:55.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 msgr2=0x7f6ed019f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 0x7f6ed019f150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006010 tx=0x7f6ec8031ee0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 shutdown_connections 2024-04-24T23:12:55.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f6ebc0a2c20 0x7f6ebc0a50e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed0103340 0x7f6ed019f150 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ed0102980 0x7f6ed01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 --2- 172.21.15.112:0/3421260968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed0101790 0x7f6ed0197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 >> 172.21.15.112:0/3421260968 conn(0x7f6ed00fcd20 msgr2=0x7f6ed0105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:55.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 shutdown_connections 2024-04-24T23:12:55.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.268+0000 7f6ed7b90640 1 -- 172.21.15.112:0/3421260968 wait complete. 2024-04-24T23:12:55.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-24T23:12:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 -- 172.21.15.112:0/555478667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800101890 msgr2=0x7f0800101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 --2- 172.21.15.112:0/555478667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800101890 0x7f0800101d10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009900 tx=0x7f07e802a410 comp rx=0 tx=0).stop 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 -- 172.21.15.112:0/555478667 shutdown_connections 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 --2- 172.21.15.112:0/555478667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0800102250 0x7f0800108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 --2- 172.21.15.112:0/555478667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0800101890 0x7f0800101d10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 --2- 172.21.15.112:0/555478667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08001006a0 0x7f0800100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 -- 172.21.15.112:0/555478667 >> 172.21.15.112:0/555478667 conn(0x7f08000fbc30 msgr2=0x7f08000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 -- 172.21.15.112:0/555478667 shutdown_connections 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.381+0000 7f0804a15640 1 -- 172.21.15.112:0/555478667 wait complete. 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f0804a15640 1 Processor -- start 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f0804a15640 1 -- start start 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f0804a15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001006a0 0x7f0800197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f0804a15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0800101890 0x7f0800198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f0804a15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0800102250 0x7f080019f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f0804a15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f080010c4d0 con 0x7f08001006a0 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f0804a15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f080010c350 con 0x7f0800101890 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f0804a15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f080010c650 con 0x7f0800102250 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001006a0 0x7f0800197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fe59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001006a0 0x7f0800197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37940/0 (socket says 172.21.15.112:37940) 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fe59c640 1 -- 172.21.15.112:0/199220629 learned_addr learned my addr 172.21.15.112:0/199220629 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07f7fff640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0800101890 0x7f0800198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fed9d640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0800102250 0x7f080019f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fe59c640 1 -- 172.21.15.112:0/199220629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0800102250 msgr2=0x7f080019f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fe59c640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0800102250 0x7f080019f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fe59c640 1 -- 172.21.15.112:0/199220629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0800101890 msgr2=0x7f0800198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fe59c640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0800101890 0x7f0800198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fe59c640 1 -- 172.21.15.112:0/199220629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080019f850 con 0x7f08001006a0 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.382+0000 7f07fed9d640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0800102250 0x7f080019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.383+0000 7f07fe59c640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001006a0 0x7f0800197c40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07f000b450 tx=0x7f07f000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.383+0000 7f07f77fe640 1 -- 172.21.15.112:0/199220629 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0015020 con 0x7f08001006a0 2024-04-24T23:12:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.383+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080019fb40 con 0x7f08001006a0 2024-04-24T23:12:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.383+0000 7f07f77fe640 1 -- 172.21.15.112:0/199220629 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07f0011d90 con 0x7f08001006a0 2024-04-24T23:12:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.383+0000 7f07f77fe640 1 -- 172.21.15.112:0/199220629 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f000f340 con 0x7f08001006a0 2024-04-24T23:12:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.383+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001a0090 con 0x7f08001006a0 2024-04-24T23:12:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.384+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08001a0260 con 0x7f08001006a0 2024-04-24T23:12:55.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.385+0000 7f07f77fe640 1 -- 172.21.15.112:0/199220629 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f07f0018020 con 0x7f08001006a0 2024-04-24T23:12:55.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.385+0000 7f07f77fe640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.386+0000 7f07f7fff640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.386+0000 7f07f77fe640 1 -- 172.21.15.112:0/199220629 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f07f00cee90 con 0x7f08001006a0 2024-04-24T23:12:55.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.386+0000 7f07f7fff640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006010 tx=0x7f07e8031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:55.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.390+0000 7f07f77fe640 1 -- 172.21.15.112:0/199220629 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f07f001f020 con 0x7f08001006a0 2024-04-24T23:12:55.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.593+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f08000fddc0 con 0x7f07ec0a2c20 2024-04-24T23:12:55.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.598+0000 7f07f77fe640 1 -- 172.21.15.112:0/199220629 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f08000fddc0 con 0x7f07ec0a2c20 2024-04-24T23:12:55.600 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_1/dfd928e6-5bb8-4127-82af-d599ace70756 2024-04-24T23:12:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07ec0a2c20 msgr2=0x7f07ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f07e8006010 tx=0x7f07e8031640 comp rx=0 tx=0).stop 2024-04-24T23:12:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001006a0 msgr2=0x7f0800197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001006a0 0x7f0800197c40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07f000b450 tx=0x7f07f000b910 comp rx=0 tx=0).stop 2024-04-24T23:12:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 shutdown_connections 2024-04-24T23:12:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07ec0a2c20 0x7f07ec0a50e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0800102250 0x7f080019f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0800101890 0x7f0800198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 --2- 172.21.15.112:0/199220629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001006a0 0x7f0800197c40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 >> 172.21.15.112:0/199220629 conn(0x7f08000fbc30 msgr2=0x7f08000fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 shutdown_connections 2024-04-24T23:12:55.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.600+0000 7f0804a15640 1 -- 172.21.15.112:0/199220629 wait complete. 2024-04-24T23:12:55.612 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.707+0000 7f644799e640 1 -- 172.21.15.112:0/3006481670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440101870 msgr2=0x7f6440101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.707+0000 7f644799e640 1 --2- 172.21.15.112:0/3006481670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440101870 0x7f6440101cf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6434009900 tx=0x7f643402a410 comp rx=0 tx=0).stop 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.707+0000 7f644799e640 1 -- 172.21.15.112:0/3006481670 shutdown_connections 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.707+0000 7f644799e640 1 --2- 172.21.15.112:0/3006481670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440102230 0x7f6440108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.707+0000 7f644799e640 1 --2- 172.21.15.112:0/3006481670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440101870 0x7f6440101cf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.707+0000 7f644799e640 1 --2- 172.21.15.112:0/3006481670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6440100680 0x7f6440100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.707+0000 7f644799e640 1 -- 172.21.15.112:0/3006481670 >> 172.21.15.112:0/3006481670 conn(0x7f64400fbbf0 msgr2=0x7f64400fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 -- 172.21.15.112:0/3006481670 shutdown_connections 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 -- 172.21.15.112:0/3006481670 wait complete. 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 Processor -- start 2024-04-24T23:12:55.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 -- start start 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6440100680 0x7f6440069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101870 0x7f644019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440102230 0x7f644019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c430 con 0x7f6440102230 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c2b0 con 0x7f6440100680 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644799e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f644010c5b0 con 0x7f6440101870 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f6445f3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440102230 0x7f644019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f6445f3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440102230 0x7f644019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37944/0 (socket says 172.21.15.112:37944) 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f644573a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6440100680 0x7f6440069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f6445f3b640 1 -- 172.21.15.112:0/763080509 learned_addr learned my addr 172.21.15.112:0/763080509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.708+0000 7f6444f39640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101870 0x7f644019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f6445f3b640 1 -- 172.21.15.112:0/763080509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101870 msgr2=0x7f644019c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f6445f3b640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101870 0x7f644019c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f6445f3b640 1 -- 172.21.15.112:0/763080509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6440100680 msgr2=0x7f6440069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f6445f3b640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6440100680 0x7f6440069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f6445f3b640 1 -- 172.21.15.112:0/763080509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644019f810 con 0x7f6440102230 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f6444f39640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101870 0x7f644019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f644573a640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6440100680 0x7f6440069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f6445f3b640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440102230 0x7f644019ef00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f643800eaa0 tx=0x7f643800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f64327fc640 1 -- 172.21.15.112:0/763080509 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643800cdb0 con 0x7f6440102230 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644019faa0 con 0x7f6440102230 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f64327fc640 1 -- 172.21.15.112:0/763080509 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6438004b20 con 0x7f6440102230 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f64327fc640 1 -- 172.21.15.112:0/763080509 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438011760 con 0x7f6440102230 2024-04-24T23:12:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.709+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644019fff0 con 0x7f6440102230 2024-04-24T23:12:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.710+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6440107940 con 0x7f6440102230 2024-04-24T23:12:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.711+0000 7f64327fc640 1 -- 172.21.15.112:0/763080509 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6438011900 con 0x7f6440102230 2024-04-24T23:12:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.711+0000 7f64327fc640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f642c0a2c20 0x7f642c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.712+0000 7f644573a640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f642c0a2c20 0x7f642c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.712+0000 7f64327fc640 1 -- 172.21.15.112:0/763080509 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f64380cf660 con 0x7f6440102230 2024-04-24T23:12:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.712+0000 7f644573a640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f642c0a2c20 0x7f642c0a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64401016d0 tx=0x7f643c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.716+0000 7f64327fc640 1 -- 172.21.15.112:0/763080509 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6438016030 con 0x7f6440102230 2024-04-24T23:12:55.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.933+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f644002d170 con 0x7f6440102230 2024-04-24T23:12:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.934+0000 7f64327fc640 1 -- 172.21.15.112:0/763080509 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11947 (secure 0 0 0) 0x7f643809c220 con 0x7f6440102230 2024-04-24T23:12:55.936 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:55.936 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:12:55.876908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24482,24490,24425,24424]},"id":1}]} 2024-04-24T23:12:55.936 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-04-24T23:12:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f642c0a2c20 msgr2=0x7f642c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f642c0a2c20 0x7f642c0a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64401016d0 tx=0x7f643c005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:55.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440102230 msgr2=0x7f644019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440102230 0x7f644019ef00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f643800eaa0 tx=0x7f643800ef60 comp rx=0 tx=0).stop 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 shutdown_connections 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f642c0a2c20 0x7f642c0a50e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6440102230 0x7f644019ef00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6440101870 0x7f644019c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 --2- 172.21.15.112:0/763080509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6440100680 0x7f6440069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 >> 172.21.15.112:0/763080509 conn(0x7f64400fbbf0 msgr2=0x7f6440104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 shutdown_connections 2024-04-24T23:12:55.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:55.936+0000 7f644799e640 1 -- 172.21.15.112:0/763080509 wait complete. 2024-04-24T23:12:55.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.047+0000 7fc29c7ee640 1 -- 172.21.15.112:0/1259582072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294100680 msgr2=0x7fc294100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.047+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/1259582072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294100680 0x7fc294100aa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc290009900 tx=0x7fc29002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.047+0000 7fc29c7ee640 1 -- 172.21.15.112:0/1259582072 shutdown_connections 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.047+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/1259582072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc294102230 0x7fc294108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.047+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/1259582072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc294101870 0x7fc294101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.047+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/1259582072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294100680 0x7fc294100aa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.047+0000 7fc29c7ee640 1 -- 172.21.15.112:0/1259582072 >> 172.21.15.112:0/1259582072 conn(0x7fc2940fbbf0 msgr2=0x7fc2940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 -- 172.21.15.112:0/1259582072 shutdown_connections 2024-04-24T23:12:56.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 -- 172.21.15.112:0/1259582072 wait complete. 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 Processor -- start 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 -- start start 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc294100680 0x7fc294069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc294101870 0x7fc29419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294102230 0x7fc29419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc29410c430 con 0x7fc294102230 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc29410c2b0 con 0x7fc294101870 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29c7ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc29410c5b0 con 0x7fc294100680 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29ad8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294102230 0x7fc29419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29ad8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294102230 0x7fc29419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37972/0 (socket says 172.21.15.112:37972) 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29ad8b640 1 -- 172.21.15.112:0/3840977061 learned_addr learned my addr 172.21.15.112:0/3840977061 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc299d89640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc294101870 0x7fc29419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.048+0000 7fc29a58a640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc294100680 0x7fc294069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29ad8b640 1 -- 172.21.15.112:0/3840977061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc294100680 msgr2=0x7fc294069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29ad8b640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc294100680 0x7fc294069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29ad8b640 1 -- 172.21.15.112:0/3840977061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc294101870 msgr2=0x7fc29419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29ad8b640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc294101870 0x7fc29419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29ad8b640 1 -- 172.21.15.112:0/3840977061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29419f800 con 0x7fc294102230 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29a58a640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc294100680 0x7fc294069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29ad8b640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294102230 0x7fc29419eef0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00cad0 tx=0x7fc28c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc28b7fe640 1 -- 172.21.15.112:0/3840977061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00ae20 con 0x7fc294102230 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc28b7fe640 1 -- 172.21.15.112:0/3840977061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc28c004b20 con 0x7fc294102230 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29419fa90 con 0x7fc294102230 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc28b7fe640 1 -- 172.21.15.112:0/3840977061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00f780 con 0x7fc294102230 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.049+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29419ff90 con 0x7fc294102230 2024-04-24T23:12:56.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.050+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2941079a0 con 0x7fc294102230 2024-04-24T23:12:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.051+0000 7fc28b7fe640 1 -- 172.21.15.112:0/3840977061 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc28c004cd0 con 0x7fc294102230 2024-04-24T23:12:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.051+0000 7fc28b7fe640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2800a2c20 0x7fc2800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.052+0000 7fc29a58a640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2800a2c20 0x7fc2800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.052+0000 7fc28b7fe640 1 -- 172.21.15.112:0/3840977061 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc28c0cf4b0 con 0x7fc294102230 2024-04-24T23:12:56.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.052+0000 7fc29a58a640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2800a2c20 0x7fc2800a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc290006010 tx=0x7fc290031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:56.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.056+0000 7fc28b7fe640 1 -- 172.21.15.112:0/3840977061 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc28c014030 con 0x7fc294102230 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] up:active 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: Cluster is now healthy 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-24T23:12:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-24T23:12:56.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-24T23:12:56.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:12:56.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:55 smithi187 ceph-mon[33292]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.235+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc29402d3c0 con 0x7fc294102230 2024-04-24T23:12:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.236+0000 7fc28b7fe640 1 -- 172.21.15.112:0/3840977061 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7fc28c09c070 con 0x7fc294102230 2024-04-24T23:12:56.239 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:56.239 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:12:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2800a2c20 msgr2=0x7fc2800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2800a2c20 0x7fc2800a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc290006010 tx=0x7fc290031640 comp rx=0 tx=0).stop 2024-04-24T23:12:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294102230 msgr2=0x7fc29419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294102230 0x7fc29419eef0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00cad0 tx=0x7fc28c00cf90 comp rx=0 tx=0).stop 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 shutdown_connections 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2800a2c20 0x7fc2800a50e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc294102230 0x7fc29419eef0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc294101870 0x7fc29419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 --2- 172.21.15.112:0/3840977061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc294100680 0x7fc294069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 >> 172.21.15.112:0/3840977061 conn(0x7fc2940fbbf0 msgr2=0x7fc294104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 shutdown_connections 2024-04-24T23:12:56.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.238+0000 7fc29c7ee640 1 -- 172.21.15.112:0/3840977061 wait complete. 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] up:active 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: Cluster is now healthy 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:12:56.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:55 smithi112 ceph-mon[29186]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.252 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-24T23:12:56.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:12:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-24T23:12:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] up:active 2024-04-24T23:12:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-24T23:12:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:12:56.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: Cluster is now healthy 2024-04-24T23:12:56.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-24T23:12:56.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-24T23:12:56.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-24T23:12:56.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-24T23:12:56.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:55 smithi150 ceph-mon[32247]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 -- 172.21.15.112:0/336781260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 msgr2=0x7f77bc0ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 --2- 172.21.15.112:0/336781260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 0x7f77bc0ffea0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009930 tx=0x7f77ac02a420 comp rx=0 tx=0).stop 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 -- 172.21.15.112:0/336781260 shutdown_connections 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 --2- 172.21.15.112:0/336781260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77bc1034b0 0x7f77bc109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 --2- 172.21.15.112:0/336781260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77bc102af0 0x7f77bc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 --2- 172.21.15.112:0/336781260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 0x7f77bc0ffea0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 -- 172.21.15.112:0/336781260 >> 172.21.15.112:0/336781260 conn(0x7f77bc0fb3b0 msgr2=0x7f77bc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 -- 172.21.15.112:0/336781260 shutdown_connections 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.347+0000 7f77c2589640 1 -- 172.21.15.112:0/336781260 wait complete. 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77c2589640 1 Processor -- start 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77c2589640 1 -- start start 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77c2589640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 0x7f77bc197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77c2589640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77bc102af0 0x7f77bc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77c2589640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77bc1034b0 0x7f77bc19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77c2589640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc10d560 con 0x7f77bc0ffa80 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77c2589640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc10d3e0 con 0x7f77bc102af0 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77c2589640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc10d6e0 con 0x7f77bc1034b0 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 0x7f77bc197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 0x7f77bc197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38004/0 (socket says 172.21.15.112:38004) 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bbfff640 1 -- 172.21.15.112:0/1441179194 learned_addr learned my addr 172.21.15.112:0/1441179194 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bb7fe640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77bc102af0 0x7f77bc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bbfff640 1 -- 172.21.15.112:0/1441179194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77bc1034b0 msgr2=0x7f77bc19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bbfff640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77bc1034b0 0x7f77bc19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bbfff640 1 -- 172.21.15.112:0/1441179194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77bc102af0 msgr2=0x7f77bc1980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bbfff640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77bc102af0 0x7f77bc1980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.348+0000 7f77bbfff640 1 -- 172.21.15.112:0/1441179194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc19f7d0 con 0x7f77bc0ffa80 2024-04-24T23:12:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.349+0000 7f77bbfff640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 0x7f77bc197bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f77ac005ab0 tx=0x7f77ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:56.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.349+0000 7f77b97fa640 1 -- 172.21.15.112:0/1441179194 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac03f030 con 0x7f77bc0ffa80 2024-04-24T23:12:56.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.349+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc19fa60 con 0x7f77bc0ffa80 2024-04-24T23:12:56.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.349+0000 7f77b97fa640 1 -- 172.21.15.112:0/1441179194 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f77ac003d00 con 0x7f77bc0ffa80 2024-04-24T23:12:56.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.349+0000 7f77b97fa640 1 -- 172.21.15.112:0/1441179194 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac035070 con 0x7f77bc0ffa80 2024-04-24T23:12:56.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.349+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc19ffa0 con 0x7f77bc0ffa80 2024-04-24T23:12:56.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.350+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc108c20 con 0x7f77bc0ffa80 2024-04-24T23:12:56.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.351+0000 7f77b97fa640 1 -- 172.21.15.112:0/1441179194 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f77ac042030 con 0x7f77bc0ffa80 2024-04-24T23:12:56.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.351+0000 7f77b97fa640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77a40a2c20 0x7f77a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.351+0000 7f77bb7fe640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77a40a2c20 0x7f77a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.351+0000 7f77b97fa640 1 -- 172.21.15.112:0/1441179194 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f77ac0ef990 con 0x7f77bc0ffa80 2024-04-24T23:12:56.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.352+0000 7f77bb7fe640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77a40a2c20 0x7f77a40a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f77bc1990c0 tx=0x7f77b000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:56.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.356+0000 7f77b97fa640 1 -- 172.21.15.112:0/1441179194 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77ac0bc550 con 0x7f77bc0ffa80 2024-04-24T23:12:56.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.526+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77bc061b80 con 0x7f77a40a2c20 2024-04-24T23:12:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.536+0000 7f77b97fa640 1 -- 172.21.15.112:0/1441179194 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340243 (secure 0 0 0) 0x7f77bc061b80 con 0x7f77a40a2c20 2024-04-24T23:12:56.543 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:56.545 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77a40a2c20 msgr2=0x7f77a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77a40a2c20 0x7f77a40a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f77bc1990c0 tx=0x7f77b000b390 comp rx=0 tx=0).stop 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 msgr2=0x7f77bc197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 0x7f77bc197bb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f77ac005ab0 tx=0x7f77ac0042f0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 shutdown_connections 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77a40a2c20 0x7f77a40a50e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77bc1034b0 0x7f77bc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f77bc102af0 0x7f77bc1980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 --2- 172.21.15.112:0/1441179194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0ffa80 0x7f77bc197bb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 >> 172.21.15.112:0/1441179194 conn(0x7f77bc0fb3b0 msgr2=0x7f77bc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:56.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 shutdown_connections 2024-04-24T23:12:56.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.540+0000 7f77c2589640 1 -- 172.21.15.112:0/1441179194 wait complete. 2024-04-24T23:12:56.553 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-24T23:12:54.789417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":250,"num_write_kb":2959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":195,"ondisk_log_size":195,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379400,"kb_used_data":13576,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562424,"statfs":{"total":1151940427776,"available":1151551922176,"internally_reserved":0,"allocated":13901824,"data_stored":7569392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003711"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993643+0000","last_change":"2024-04-24T23:12:28.013506+0000","last_active":"2024-04-24T23:12:33.993643+0000","last_peered":"2024-04-24T23:12:33.993643+0000","last_clean":"2024-04-24T23:12:33.993643+0000","last_became_active":"2024-04-24T23:12:28.012652+0000","last_became_peered":"2024-04-24T23:12:28.012652+0000","last_unstale":"2024-04-24T23:12:33.993643+0000","last_undegraded":"2024-04-24T23:12:33.993643+0000","last_fullsized":"2024-04-24T23:12:33.993643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:57.420465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991651+0000","last_change":"2024-04-24T23:12:28.004998+0000","last_active":"2024-04-24T23:12:33.991651+0000","last_peered":"2024-04-24T23:12:33.991651+0000","last_clean":"2024-04-24T23:12:33.991651+0000","last_became_active":"2024-04-24T23:12:28.004706+0000","last_became_peered":"2024-04-24T23:12:28.004706+0000","last_unstale":"2024-04-24T23:12:33.991651+0000","last_undegraded":"2024-04-24T23:12:33.991651+0000","last_fullsized":"2024-04-24T23:12:33.991651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.759515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991865+0000","last_change":"2024-04-24T23:12:28.019170+0000","last_active":"2024-04-24T23:12:33.991865+0000","last_peered":"2024-04-24T23:12:33.991865+0000","last_clean":"2024-04-24T23:12:33.991865+0000","last_became_active":"2024-04-24T23:12:28.018815+0000","last_became_peered":"2024-04-24T23:12:28.018815+0000","last_unstale":"2024-04-24T23:12:33.991865+0000","last_undegraded":"2024-04-24T23:12:33.991865+0000","last_fullsized":"2024-04-24T23:12:33.991865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:43.830598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992683+0000","last_change":"2024-04-24T23:12:28.008662+0000","last_active":"2024-04-24T23:12:33.992683+0000","last_peered":"2024-04-24T23:12:33.992683+0000","last_clean":"2024-04-24T23:12:33.992683+0000","last_became_active":"2024-04-24T23:12:28.008282+0000","last_became_peered":"2024-04-24T23:12:28.008282+0000","last_unstale":"2024-04-24T23:12:33.992683+0000","last_undegraded":"2024-04-24T23:12:33.992683+0000","last_fullsized":"2024-04-24T23:12:33.992683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:17.558899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992177+0000","last_change":"2024-04-24T23:12:28.005212+0000","last_active":"2024-04-24T23:12:33.992177+0000","last_peered":"2024-04-24T23:12:33.992177+0000","last_clean":"2024-04-24T23:12:33.992177+0000","last_became_active":"2024-04-24T23:12:28.004853+0000","last_became_peered":"2024-04-24T23:12:28.004853+0000","last_unstale":"2024-04-24T23:12:33.992177+0000","last_undegraded":"2024-04-24T23:12:33.992177+0000","last_fullsized":"2024-04-24T23:12:33.992177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:33.650922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992649+0000","last_change":"2024-04-24T23:12:28.004546+0000","last_active":"2024-04-24T23:12:33.992649+0000","last_peered":"2024-04-24T23:12:33.992649+0000","last_clean":"2024-04-24T23:12:33.992649+0000","last_became_active":"2024-04-24T23:12:28.004194+0000","last_became_peered":"2024-04-24T23:12:28.004194+0000","last_unstale":"2024-04-24T23:12:33.992649+0000","last_undegraded":"2024-04-24T23:12:33.992649+0000","last_fullsized":"2024-04-24T23:12:33.992649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:57.772271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512359+0000","last_change":"2024-04-24T23:12:28.009436+0000","last_active":"2024-04-24T23:12:33.512359+0000","last_peered":"2024-04-24T23:12:33.512359+0000","last_clean":"2024-04-24T23:12:33.512359+0000","last_became_active":"2024-04-24T23:12:28.008219+0000","last_became_peered":"2024-04-24T23:12:28.008219+0000","last_unstale":"2024-04-24T23:12:33.512359+0000","last_undegraded":"2024-04-24T23:12:33.512359+0000","last_fullsized":"2024-04-24T23:12:33.512359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991331+0000","last_change":"2024-04-24T23:12:28.010749+0000","last_active":"2024-04-24T23:12:33.991331+0000","last_peered":"2024-04-24T23:12:33.991331+0000","last_clean":"2024-04-24T23:12:33.991331+0000","last_became_active":"2024-04-24T23:12:28.008999+0000","last_became_peered":"2024-04-24T23:12:28.008999+0000","last_unstale":"2024-04-24T23:12:33.991331+0000","last_undegraded":"2024-04-24T23:12:33.991331+0000","last_fullsized":"2024-04-24T23:12:33.991331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:23.327427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256720+0000","last_change":"2024-04-24T23:12:28.004948+0000","last_active":"2024-04-24T23:12:33.256720+0000","last_peered":"2024-04-24T23:12:33.256720+0000","last_clean":"2024-04-24T23:12:33.256720+0000","last_became_active":"2024-04-24T23:12:28.004380+0000","last_became_peered":"2024-04-24T23:12:28.004380+0000","last_unstale":"2024-04-24T23:12:33.256720+0000","last_undegraded":"2024-04-24T23:12:33.256720+0000","last_fullsized":"2024-04-24T23:12:33.256720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:53.999210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993173+0000","last_change":"2024-04-24T23:12:28.005192+0000","last_active":"2024-04-24T23:12:33.993173+0000","last_peered":"2024-04-24T23:12:33.993173+0000","last_clean":"2024-04-24T23:12:33.993173+0000","last_became_active":"2024-04-24T23:12:28.004860+0000","last_became_peered":"2024-04-24T23:12:28.004860+0000","last_unstale":"2024-04-24T23:12:33.993173+0000","last_undegraded":"2024-04-24T23:12:33.993173+0000","last_fullsized":"2024-04-24T23:12:33.993173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:21.874408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512846+0000","last_change":"2024-04-24T23:12:28.016098+0000","last_active":"2024-04-24T23:12:33.512846+0000","last_peered":"2024-04-24T23:12:33.512846+0000","last_clean":"2024-04-24T23:12:33.512846+0000","last_became_active":"2024-04-24T23:12:28.015736+0000","last_became_peered":"2024-04-24T23:12:28.015736+0000","last_unstale":"2024-04-24T23:12:33.512846+0000","last_undegraded":"2024-04-24T23:12:33.512846+0000","last_fullsized":"2024-04-24T23:12:33.512846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.071797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992553+0000","last_change":"2024-04-24T23:12:28.010651+0000","last_active":"2024-04-24T23:12:33.992553+0000","last_peered":"2024-04-24T23:12:33.992553+0000","last_clean":"2024-04-24T23:12:33.992553+0000","last_became_active":"2024-04-24T23:12:28.009700+0000","last_became_peered":"2024-04-24T23:12:28.009700+0000","last_unstale":"2024-04-24T23:12:33.992553+0000","last_undegraded":"2024-04-24T23:12:33.992553+0000","last_fullsized":"2024-04-24T23:12:33.992553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:26.172386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253853+0000","last_change":"2024-04-24T23:12:28.005743+0000","last_active":"2024-04-24T23:12:33.253853+0000","last_peered":"2024-04-24T23:12:33.253853+0000","last_clean":"2024-04-24T23:12:33.253853+0000","last_became_active":"2024-04-24T23:12:28.005450+0000","last_became_peered":"2024-04-24T23:12:28.005450+0000","last_unstale":"2024-04-24T23:12:33.253853+0000","last_undegraded":"2024-04-24T23:12:33.253853+0000","last_fullsized":"2024-04-24T23:12:33.253853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:14.858212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991308+0000","last_change":"2024-04-24T23:12:28.004140+0000","last_active":"2024-04-24T23:12:33.991308+0000","last_peered":"2024-04-24T23:12:33.991308+0000","last_clean":"2024-04-24T23:12:33.991308+0000","last_became_active":"2024-04-24T23:12:28.003794+0000","last_became_peered":"2024-04-24T23:12:28.003794+0000","last_unstale":"2024-04-24T23:12:33.991308+0000","last_undegraded":"2024-04-24T23:12:33.991308+0000","last_fullsized":"2024-04-24T23:12:33.991308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:02.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991361+0000","last_change":"2024-04-24T23:12:28.006834+0000","last_active":"2024-04-24T23:12:33.991361+0000","last_peered":"2024-04-24T23:12:33.991361+0000","last_clean":"2024-04-24T23:12:33.991361+0000","last_became_active":"2024-04-24T23:12:28.006504+0000","last_became_peered":"2024-04-24T23:12:28.006504+0000","last_unstale":"2024-04-24T23:12:33.991361+0000","last_undegraded":"2024-04-24T23:12:33.991361+0000","last_fullsized":"2024-04-24T23:12:33.991361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:21.194464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256036+0000","last_change":"2024-04-24T23:12:28.013132+0000","last_active":"2024-04-24T23:12:33.256036+0000","last_peered":"2024-04-24T23:12:33.256036+0000","last_clean":"2024-04-24T23:12:33.256036+0000","last_became_active":"2024-04-24T23:12:28.012751+0000","last_became_peered":"2024-04-24T23:12:28.012751+0000","last_unstale":"2024-04-24T23:12:33.256036+0000","last_undegraded":"2024-04-24T23:12:33.256036+0000","last_fullsized":"2024-04-24T23:12:33.256036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.942985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993478+0000","last_change":"2024-04-24T23:12:28.013719+0000","last_active":"2024-04-24T23:12:33.993478+0000","last_peered":"2024-04-24T23:12:33.993478+0000","last_clean":"2024-04-24T23:12:33.993478+0000","last_became_active":"2024-04-24T23:12:28.013108+0000","last_became_peered":"2024-04-24T23:12:28.013108+0000","last_unstale":"2024-04-24T23:12:33.993478+0000","last_undegraded":"2024-04-24T23:12:33.993478+0000","last_fullsized":"2024-04-24T23:12:33.993478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:38.248867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993476+0000","last_change":"2024-04-24T23:12:28.012386+0000","last_active":"2024-04-24T23:12:33.993476+0000","last_peered":"2024-04-24T23:12:33.993476+0000","last_clean":"2024-04-24T23:12:33.993476+0000","last_became_active":"2024-04-24T23:12:28.011340+0000","last_became_peered":"2024-04-24T23:12:28.011340+0000","last_unstale":"2024-04-24T23:12:33.993476+0000","last_undegraded":"2024-04-24T23:12:33.993476+0000","last_fullsized":"2024-04-24T23:12:33.993476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:48.115600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993007+0000","last_change":"2024-04-24T23:12:28.004033+0000","last_active":"2024-04-24T23:12:33.993007+0000","last_peered":"2024-04-24T23:12:33.993007+0000","last_clean":"2024-04-24T23:12:33.993007+0000","last_became_active":"2024-04-24T23:12:28.003648+0000","last_became_peered":"2024-04-24T23:12:28.003648+0000","last_unstale":"2024-04-24T23:12:33.993007+0000","last_undegraded":"2024-04-24T23:12:33.993007+0000","last_fullsized":"2024-04-24T23:12:33.993007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:13.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993074+0000","last_change":"2024-04-24T23:12:32.026590+0000","last_active":"2024-04-24T23:12:33.993074+0000","last_peered":"2024-04-24T23:12:33.993074+0000","last_clean":"2024-04-24T23:12:33.993074+0000","last_became_active":"2024-04-24T23:12:32.026098+0000","last_became_peered":"2024-04-24T23:12:32.026098+0000","last_unstale":"2024-04-24T23:12:33.993074+0000","last_undegraded":"2024-04-24T23:12:33.993074+0000","last_fullsized":"2024-04-24T23:12:33.993074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:55.812641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993122+0000","last_change":"2024-04-24T23:12:27.005325+0000","last_active":"2024-04-24T23:12:33.993122+0000","last_peered":"2024-04-24T23:12:33.993122+0000","last_clean":"2024-04-24T23:12:33.993122+0000","last_became_active":"2024-04-24T23:12:27.004627+0000","last_became_peered":"2024-04-24T23:12:27.004627+0000","last_unstale":"2024-04-24T23:12:33.993122+0000","last_undegraded":"2024-04-24T23:12:33.993122+0000","last_fullsized":"2024-04-24T23:12:33.993122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.985647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992318+0000","last_change":"2024-04-24T23:12:28.007872+0000","last_active":"2024-04-24T23:12:33.992318+0000","last_peered":"2024-04-24T23:12:33.992318+0000","last_clean":"2024-04-24T23:12:33.992318+0000","last_became_active":"2024-04-24T23:12:28.007493+0000","last_became_peered":"2024-04-24T23:12:28.007493+0000","last_unstale":"2024-04-24T23:12:33.992318+0000","last_undegraded":"2024-04-24T23:12:33.992318+0000","last_fullsized":"2024-04-24T23:12:33.992318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:26.686831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991618+0000","last_change":"2024-04-24T23:12:32.031138+0000","last_active":"2024-04-24T23:12:33.991618+0000","last_peered":"2024-04-24T23:12:33.991618+0000","last_clean":"2024-04-24T23:12:33.991618+0000","last_became_active":"2024-04-24T23:12:32.030626+0000","last_became_peered":"2024-04-24T23:12:32.030626+0000","last_unstale":"2024-04-24T23:12:33.991618+0000","last_undegraded":"2024-04-24T23:12:33.991618+0000","last_fullsized":"2024-04-24T23:12:33.991618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992668+0000","last_change":"2024-04-24T23:12:26.999954+0000","last_active":"2024-04-24T23:12:33.992668+0000","last_peered":"2024-04-24T23:12:33.992668+0000","last_clean":"2024-04-24T23:12:33.992668+0000","last_became_active":"2024-04-24T23:12:26.999507+0000","last_became_peered":"2024-04-24T23:12:26.999507+0000","last_unstale":"2024-04-24T23:12:33.992668+0000","last_undegraded":"2024-04-24T23:12:33.992668+0000","last_fullsized":"2024-04-24T23:12:33.992668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:23.984002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991739+0000","last_change":"2024-04-24T23:12:28.019342+0000","last_active":"2024-04-24T23:12:33.991739+0000","last_peered":"2024-04-24T23:12:33.991739+0000","last_clean":"2024-04-24T23:12:33.991739+0000","last_became_active":"2024-04-24T23:12:28.018976+0000","last_became_peered":"2024-04-24T23:12:28.018976+0000","last_unstale":"2024-04-24T23:12:33.991739+0000","last_undegraded":"2024-04-24T23:12:33.991739+0000","last_fullsized":"2024-04-24T23:12:33.991739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:18.737495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992389+0000","last_change":"2024-04-24T23:12:32.031015+0000","last_active":"2024-04-24T23:12:33.992389+0000","last_peered":"2024-04-24T23:12:33.992389+0000","last_clean":"2024-04-24T23:12:33.992389+0000","last_became_active":"2024-04-24T23:12:32.030564+0000","last_became_peered":"2024-04-24T23:12:32.030564+0000","last_unstale":"2024-04-24T23:12:33.992389+0000","last_undegraded":"2024-04-24T23:12:33.992389+0000","last_fullsized":"2024-04-24T23:12:33.992389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:12.247397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993110+0000","last_change":"2024-04-24T23:12:27.001229+0000","last_active":"2024-04-24T23:12:33.993110+0000","last_peered":"2024-04-24T23:12:33.993110+0000","last_clean":"2024-04-24T23:12:33.993110+0000","last_became_active":"2024-04-24T23:12:27.000889+0000","last_became_peered":"2024-04-24T23:12:27.000889+0000","last_unstale":"2024-04-24T23:12:33.993110+0000","last_undegraded":"2024-04-24T23:12:33.993110+0000","last_fullsized":"2024-04-24T23:12:33.993110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:39.189828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256356+0000","last_change":"2024-04-24T23:12:28.012309+0000","last_active":"2024-04-24T23:12:33.256356+0000","last_peered":"2024-04-24T23:12:33.256356+0000","last_clean":"2024-04-24T23:12:33.256356+0000","last_became_active":"2024-04-24T23:12:28.011537+0000","last_became_peered":"2024-04-24T23:12:28.011537+0000","last_unstale":"2024-04-24T23:12:33.256356+0000","last_undegraded":"2024-04-24T23:12:33.256356+0000","last_fullsized":"2024-04-24T23:12:33.256356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:38.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991619+0000","last_change":"2024-04-24T23:12:32.030830+0000","last_active":"2024-04-24T23:12:33.991619+0000","last_peered":"2024-04-24T23:12:33.991619+0000","last_clean":"2024-04-24T23:12:33.991619+0000","last_became_active":"2024-04-24T23:12:32.030318+0000","last_became_peered":"2024-04-24T23:12:32.030318+0000","last_unstale":"2024-04-24T23:12:33.991619+0000","last_undegraded":"2024-04-24T23:12:33.991619+0000","last_fullsized":"2024-04-24T23:12:33.991619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:25.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992350+0000","last_change":"2024-04-24T23:12:26.997926+0000","last_active":"2024-04-24T23:12:33.992350+0000","last_peered":"2024-04-24T23:12:33.992350+0000","last_clean":"2024-04-24T23:12:33.992350+0000","last_became_active":"2024-04-24T23:12:26.997552+0000","last_became_peered":"2024-04-24T23:12:26.997552+0000","last_unstale":"2024-04-24T23:12:33.992350+0000","last_undegraded":"2024-04-24T23:12:33.992350+0000","last_fullsized":"2024-04-24T23:12:33.992350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:50.926624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252316+0000","last_change":"2024-04-24T23:12:28.008486+0000","last_active":"2024-04-24T23:12:33.252316+0000","last_peered":"2024-04-24T23:12:33.252316+0000","last_clean":"2024-04-24T23:12:33.252316+0000","last_became_active":"2024-04-24T23:12:28.008150+0000","last_became_peered":"2024-04-24T23:12:28.008150+0000","last_unstale":"2024-04-24T23:12:33.252316+0000","last_undegraded":"2024-04-24T23:12:33.252316+0000","last_fullsized":"2024-04-24T23:12:33.252316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:15.935567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992845+0000","last_change":"2024-04-24T23:12:32.021601+0000","last_active":"2024-04-24T23:12:33.992845+0000","last_peered":"2024-04-24T23:12:33.992845+0000","last_clean":"2024-04-24T23:12:33.992845+0000","last_became_active":"2024-04-24T23:12:32.021141+0000","last_became_peered":"2024-04-24T23:12:32.021141+0000","last_unstale":"2024-04-24T23:12:33.992845+0000","last_undegraded":"2024-04-24T23:12:33.992845+0000","last_fullsized":"2024-04-24T23:12:33.992845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:53.795337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252082+0000","last_change":"2024-04-24T23:12:27.002639+0000","last_active":"2024-04-24T23:12:33.252082+0000","last_peered":"2024-04-24T23:12:33.252082+0000","last_clean":"2024-04-24T23:12:33.252082+0000","last_became_active":"2024-04-24T23:12:27.002295+0000","last_became_peered":"2024-04-24T23:12:27.002295+0000","last_unstale":"2024-04-24T23:12:33.252082+0000","last_undegraded":"2024-04-24T23:12:33.252082+0000","last_fullsized":"2024-04-24T23:12:33.252082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:40.962878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992430+0000","last_change":"2024-04-24T23:12:28.000123+0000","last_active":"2024-04-24T23:12:33.992430+0000","last_peered":"2024-04-24T23:12:33.992430+0000","last_clean":"2024-04-24T23:12:33.992430+0000","last_became_active":"2024-04-24T23:12:27.999778+0000","last_became_peered":"2024-04-24T23:12:27.999778+0000","last_unstale":"2024-04-24T23:12:33.992430+0000","last_undegraded":"2024-04-24T23:12:33.992430+0000","last_fullsized":"2024-04-24T23:12:33.992430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:03.827336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992635+0000","last_change":"2024-04-24T23:12:32.023886+0000","last_active":"2024-04-24T23:12:33.992635+0000","last_peered":"2024-04-24T23:12:33.992635+0000","last_clean":"2024-04-24T23:12:33.992635+0000","last_became_active":"2024-04-24T23:12:32.022796+0000","last_became_peered":"2024-04-24T23:12:32.022796+0000","last_unstale":"2024-04-24T23:12:33.992635+0000","last_undegraded":"2024-04-24T23:12:33.992635+0000","last_fullsized":"2024-04-24T23:12:33.992635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:17.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:52.899756+0000","last_change":"2024-04-24T23:12:26.999210+0000","last_active":"2024-04-24T23:12:52.899756+0000","last_peered":"2024-04-24T23:12:52.899756+0000","last_clean":"2024-04-24T23:12:52.899756+0000","last_became_active":"2024-04-24T23:12:26.998642+0000","last_became_peered":"2024-04-24T23:12:26.998642+0000","last_unstale":"2024-04-24T23:12:52.899756+0000","last_undegraded":"2024-04-24T23:12:52.899756+0000","last_fullsized":"2024-04-24T23:12:52.899756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:39.171955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252347+0000","last_change":"2024-04-24T23:12:28.006903+0000","last_active":"2024-04-24T23:12:33.252347+0000","last_peered":"2024-04-24T23:12:33.252347+0000","last_clean":"2024-04-24T23:12:33.252347+0000","last_became_active":"2024-04-24T23:12:28.006541+0000","last_became_peered":"2024-04-24T23:12:28.006541+0000","last_unstale":"2024-04-24T23:12:33.252347+0000","last_undegraded":"2024-04-24T23:12:33.252347+0000","last_fullsized":"2024-04-24T23:12:33.252347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:04.739082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993465+0000","last_change":"2024-04-24T23:12:32.028192+0000","last_active":"2024-04-24T23:12:33.993465+0000","last_peered":"2024-04-24T23:12:33.993465+0000","last_clean":"2024-04-24T23:12:33.993465+0000","last_became_active":"2024-04-24T23:12:32.027715+0000","last_became_peered":"2024-04-24T23:12:32.027715+0000","last_unstale":"2024-04-24T23:12:33.993465+0000","last_undegraded":"2024-04-24T23:12:33.993465+0000","last_fullsized":"2024-04-24T23:12:33.993465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:22.577980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253650+0000","last_change":"2024-04-24T23:12:27.000212+0000","last_active":"2024-04-24T23:12:33.253650+0000","last_peered":"2024-04-24T23:12:33.253650+0000","last_clean":"2024-04-24T23:12:33.253650+0000","last_became_active":"2024-04-24T23:12:26.999844+0000","last_became_peered":"2024-04-24T23:12:26.999844+0000","last_unstale":"2024-04-24T23:12:33.253650+0000","last_undegraded":"2024-04-24T23:12:33.253650+0000","last_fullsized":"2024-04-24T23:12:33.253650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:00.837670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991766+0000","last_change":"2024-04-24T23:12:28.002875+0000","last_active":"2024-04-24T23:12:33.991766+0000","last_peered":"2024-04-24T23:12:33.991766+0000","last_clean":"2024-04-24T23:12:33.991766+0000","last_became_active":"2024-04-24T23:12:28.002572+0000","last_became_peered":"2024-04-24T23:12:28.002572+0000","last_unstale":"2024-04-24T23:12:33.991766+0000","last_undegraded":"2024-04-24T23:12:33.991766+0000","last_fullsized":"2024-04-24T23:12:33.991766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:55.931371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991540+0000","last_change":"2024-04-24T23:12:32.024729+0000","last_active":"2024-04-24T23:12:33.991540+0000","last_peered":"2024-04-24T23:12:33.991540+0000","last_clean":"2024-04-24T23:12:33.991540+0000","last_became_active":"2024-04-24T23:12:32.024316+0000","last_became_peered":"2024-04-24T23:12:32.024316+0000","last_unstale":"2024-04-24T23:12:33.991540+0000","last_undegraded":"2024-04-24T23:12:33.991540+0000","last_fullsized":"2024-04-24T23:12:33.991540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:21.668782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252542+0000","last_change":"2024-04-24T23:12:27.003435+0000","last_active":"2024-04-24T23:12:33.252542+0000","last_peered":"2024-04-24T23:12:33.252542+0000","last_clean":"2024-04-24T23:12:33.252542+0000","last_became_active":"2024-04-24T23:12:27.003101+0000","last_became_peered":"2024-04-24T23:12:27.003101+0000","last_unstale":"2024-04-24T23:12:33.252542+0000","last_undegraded":"2024-04-24T23:12:33.252542+0000","last_fullsized":"2024-04-24T23:12:33.252542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:42.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992833+0000","last_change":"2024-04-24T23:12:28.005056+0000","last_active":"2024-04-24T23:12:33.992833+0000","last_peered":"2024-04-24T23:12:33.992833+0000","last_clean":"2024-04-24T23:12:33.992833+0000","last_became_active":"2024-04-24T23:12:28.004725+0000","last_became_peered":"2024-04-24T23:12:28.004725+0000","last_unstale":"2024-04-24T23:12:33.992833+0000","last_undegraded":"2024-04-24T23:12:33.992833+0000","last_fullsized":"2024-04-24T23:12:33.992833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:49.384950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992089+0000","last_change":"2024-04-24T23:12:32.032947+0000","last_active":"2024-04-24T23:12:33.992089+0000","last_peered":"2024-04-24T23:12:33.992089+0000","last_clean":"2024-04-24T23:12:33.992089+0000","last_became_active":"2024-04-24T23:12:32.032560+0000","last_became_peered":"2024-04-24T23:12:32.032560+0000","last_unstale":"2024-04-24T23:12:33.992089+0000","last_undegraded":"2024-04-24T23:12:33.992089+0000","last_fullsized":"2024-04-24T23:12:33.992089+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:29.282938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:51.888355+0000","last_change":"2024-04-24T23:12:27.000011+0000","last_active":"2024-04-24T23:12:51.888355+0000","last_peered":"2024-04-24T23:12:51.888355+0000","last_clean":"2024-04-24T23:12:51.888355+0000","last_became_active":"2024-04-24T23:12:26.999572+0000","last_became_peered":"2024-04-24T23:12:26.999572+0000","last_unstale":"2024-04-24T23:12:51.888355+0000","last_undegraded":"2024-04-24T23:12:51.888355+0000","last_fullsized":"2024-04-24T23:12:51.888355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:50.453964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256711+0000","last_change":"2024-04-24T23:12:28.012808+0000","last_active":"2024-04-24T23:12:33.256711+0000","last_peered":"2024-04-24T23:12:33.256711+0000","last_clean":"2024-04-24T23:12:33.256711+0000","last_became_active":"2024-04-24T23:12:28.012406+0000","last_became_peered":"2024-04-24T23:12:28.012406+0000","last_unstale":"2024-04-24T23:12:33.256711+0000","last_undegraded":"2024-04-24T23:12:33.256711+0000","last_fullsized":"2024-04-24T23:12:33.256711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:19.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991826+0000","last_change":"2024-04-24T23:12:32.027156+0000","last_active":"2024-04-24T23:12:33.991826+0000","last_peered":"2024-04-24T23:12:33.991826+0000","last_clean":"2024-04-24T23:12:33.991826+0000","last_became_active":"2024-04-24T23:12:32.026743+0000","last_became_peered":"2024-04-24T23:12:32.026743+0000","last_unstale":"2024-04-24T23:12:33.991826+0000","last_undegraded":"2024-04-24T23:12:33.991826+0000","last_fullsized":"2024-04-24T23:12:33.991826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:43.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512841+0000","last_change":"2024-04-24T23:12:27.001832+0000","last_active":"2024-04-24T23:12:33.512841+0000","last_peered":"2024-04-24T23:12:33.512841+0000","last_clean":"2024-04-24T23:12:33.512841+0000","last_became_active":"2024-04-24T23:12:27.001482+0000","last_became_peered":"2024-04-24T23:12:27.001482+0000","last_unstale":"2024-04-24T23:12:33.512841+0000","last_undegraded":"2024-04-24T23:12:33.512841+0000","last_fullsized":"2024-04-24T23:12:33.512841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:28.949767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991854+0000","last_change":"2024-04-24T23:12:28.010507+0000","last_active":"2024-04-24T23:12:33.991854+0000","last_peered":"2024-04-24T23:12:33.991854+0000","last_clean":"2024-04-24T23:12:33.991854+0000","last_became_active":"2024-04-24T23:12:28.010181+0000","last_became_peered":"2024-04-24T23:12:28.010181+0000","last_unstale":"2024-04-24T23:12:33.991854+0000","last_undegraded":"2024-04-24T23:12:33.991854+0000","last_fullsized":"2024-04-24T23:12:33.991854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:12.624880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991941+0000","last_change":"2024-04-24T23:12:32.023007+0000","last_active":"2024-04-24T23:12:33.991941+0000","last_peered":"2024-04-24T23:12:33.991941+0000","last_clean":"2024-04-24T23:12:33.991941+0000","last_became_active":"2024-04-24T23:12:32.021324+0000","last_became_peered":"2024-04-24T23:12:32.021324+0000","last_unstale":"2024-04-24T23:12:33.991941+0000","last_undegraded":"2024-04-24T23:12:33.991941+0000","last_fullsized":"2024-04-24T23:12:33.991941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:41.542778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252084+0000","last_change":"2024-04-24T23:12:26.990562+0000","last_active":"2024-04-24T23:12:33.252084+0000","last_peered":"2024-04-24T23:12:33.252084+0000","last_clean":"2024-04-24T23:12:33.252084+0000","last_became_active":"2024-04-24T23:12:26.990146+0000","last_became_peered":"2024-04-24T23:12:26.990146+0000","last_unstale":"2024-04-24T23:12:33.252084+0000","last_undegraded":"2024-04-24T23:12:33.252084+0000","last_fullsized":"2024-04-24T23:12:33.252084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.621051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256491+0000","last_change":"2024-04-24T23:12:28.000954+0000","last_active":"2024-04-24T23:12:33.256491+0000","last_peered":"2024-04-24T23:12:33.256491+0000","last_clean":"2024-04-24T23:12:33.256491+0000","last_became_active":"2024-04-24T23:12:28.000610+0000","last_became_peered":"2024-04-24T23:12:28.000610+0000","last_unstale":"2024-04-24T23:12:33.256491+0000","last_undegraded":"2024-04-24T23:12:33.256491+0000","last_fullsized":"2024-04-24T23:12:33.256491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:32.585919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993576+0000","last_change":"2024-04-24T23:12:32.030373+0000","last_active":"2024-04-24T23:12:33.993576+0000","last_peered":"2024-04-24T23:12:33.993576+0000","last_clean":"2024-04-24T23:12:33.993576+0000","last_became_active":"2024-04-24T23:12:32.029954+0000","last_became_peered":"2024-04-24T23:12:32.029954+0000","last_unstale":"2024-04-24T23:12:33.993576+0000","last_undegraded":"2024-04-24T23:12:33.993576+0000","last_fullsized":"2024-04-24T23:12:33.993576+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:28.061509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:51.888657+0000","last_change":"2024-04-24T23:12:26.996719+0000","last_active":"2024-04-24T23:12:51.888657+0000","last_peered":"2024-04-24T23:12:51.888657+0000","last_clean":"2024-04-24T23:12:51.888657+0000","last_became_active":"2024-04-24T23:12:26.996360+0000","last_became_peered":"2024-04-24T23:12:26.996360+0000","last_unstale":"2024-04-24T23:12:51.888657+0000","last_undegraded":"2024-04-24T23:12:51.888657+0000","last_fullsized":"2024-04-24T23:12:51.888657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.526902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992883+0000","last_change":"2024-04-24T23:12:28.002889+0000","last_active":"2024-04-24T23:12:33.992883+0000","last_peered":"2024-04-24T23:12:33.992883+0000","last_clean":"2024-04-24T23:12:33.992883+0000","last_became_active":"2024-04-24T23:12:28.002540+0000","last_became_peered":"2024-04-24T23:12:28.002540+0000","last_unstale":"2024-04-24T23:12:33.992883+0000","last_undegraded":"2024-04-24T23:12:33.992883+0000","last_fullsized":"2024-04-24T23:12:33.992883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:57.792628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252793+0000","last_change":"2024-04-24T23:12:32.031160+0000","last_active":"2024-04-24T23:12:33.252793+0000","last_peered":"2024-04-24T23:12:33.252793+0000","last_clean":"2024-04-24T23:12:33.252793+0000","last_became_active":"2024-04-24T23:12:32.030719+0000","last_became_peered":"2024-04-24T23:12:32.030719+0000","last_unstale":"2024-04-24T23:12:33.252793+0000","last_undegraded":"2024-04-24T23:12:33.252793+0000","last_fullsized":"2024-04-24T23:12:33.252793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:45.648011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992114+0000","last_change":"2024-04-24T23:12:26.998802+0000","last_active":"2024-04-24T23:12:33.992114+0000","last_peered":"2024-04-24T23:12:33.992114+0000","last_clean":"2024-04-24T23:12:33.992114+0000","last_became_active":"2024-04-24T23:12:26.998471+0000","last_became_peered":"2024-04-24T23:12:26.998471+0000","last_unstale":"2024-04-24T23:12:33.992114+0000","last_undegraded":"2024-04-24T23:12:33.992114+0000","last_fullsized":"2024-04-24T23:12:33.992114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:54.576879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253965+0000","last_change":"2024-04-24T23:12:28.003211+0000","last_active":"2024-04-24T23:12:33.253965+0000","last_peered":"2024-04-24T23:12:33.253965+0000","last_clean":"2024-04-24T23:12:33.253965+0000","last_became_active":"2024-04-24T23:12:28.002897+0000","last_became_peered":"2024-04-24T23:12:28.002897+0000","last_unstale":"2024-04-24T23:12:33.253965+0000","last_undegraded":"2024-04-24T23:12:33.253965+0000","last_fullsized":"2024-04-24T23:12:33.253965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.187239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992488+0000","last_change":"2024-04-24T23:12:32.028857+0000","last_active":"2024-04-24T23:12:33.992488+0000","last_peered":"2024-04-24T23:12:33.992488+0000","last_clean":"2024-04-24T23:12:33.992488+0000","last_became_active":"2024-04-24T23:12:32.028455+0000","last_became_peered":"2024-04-24T23:12:32.028455+0000","last_unstale":"2024-04-24T23:12:33.992488+0000","last_undegraded":"2024-04-24T23:12:33.992488+0000","last_fullsized":"2024-04-24T23:12:33.992488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:23.073895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252354+0000","last_change":"2024-04-24T23:12:27.003525+0000","last_active":"2024-04-24T23:12:33.252354+0000","last_peered":"2024-04-24T23:12:33.252354+0000","last_clean":"2024-04-24T23:12:33.252354+0000","last_became_active":"2024-04-24T23:12:27.003203+0000","last_became_peered":"2024-04-24T23:12:27.003203+0000","last_unstale":"2024-04-24T23:12:33.252354+0000","last_undegraded":"2024-04-24T23:12:33.252354+0000","last_fullsized":"2024-04-24T23:12:33.252354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:18.521925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512579+0000","last_change":"2024-04-24T23:12:28.017111+0000","last_active":"2024-04-24T23:12:33.512579+0000","last_peered":"2024-04-24T23:12:33.512579+0000","last_clean":"2024-04-24T23:12:33.512579+0000","last_became_active":"2024-04-24T23:12:28.016725+0000","last_became_peered":"2024-04-24T23:12:28.016725+0000","last_unstale":"2024-04-24T23:12:33.512579+0000","last_undegraded":"2024-04-24T23:12:33.512579+0000","last_fullsized":"2024-04-24T23:12:33.512579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:25.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992818+0000","last_change":"2024-04-24T23:12:32.027963+0000","last_active":"2024-04-24T23:12:33.992818+0000","last_peered":"2024-04-24T23:12:33.992818+0000","last_clean":"2024-04-24T23:12:33.992818+0000","last_became_active":"2024-04-24T23:12:32.027585+0000","last_became_peered":"2024-04-24T23:12:32.027585+0000","last_unstale":"2024-04-24T23:12:33.992818+0000","last_undegraded":"2024-04-24T23:12:33.992818+0000","last_fullsized":"2024-04-24T23:12:33.992818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:27.111414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991667+0000","last_change":"2024-04-24T23:12:27.002797+0000","last_active":"2024-04-24T23:12:33.991667+0000","last_peered":"2024-04-24T23:12:33.991667+0000","last_clean":"2024-04-24T23:12:33.991667+0000","last_became_active":"2024-04-24T23:12:27.002086+0000","last_became_peered":"2024-04-24T23:12:27.002086+0000","last_unstale":"2024-04-24T23:12:33.991667+0000","last_undegraded":"2024-04-24T23:12:33.991667+0000","last_fullsized":"2024-04-24T23:12:33.991667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:55.025453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512401+0000","last_change":"2024-04-24T23:12:28.014047+0000","last_active":"2024-04-24T23:12:33.512401+0000","last_peered":"2024-04-24T23:12:33.512401+0000","last_clean":"2024-04-24T23:12:33.512401+0000","last_became_active":"2024-04-24T23:12:28.013699+0000","last_became_peered":"2024-04-24T23:12:28.013699+0000","last_unstale":"2024-04-24T23:12:33.512401+0000","last_undegraded":"2024-04-24T23:12:33.512401+0000","last_fullsized":"2024-04-24T23:12:33.512401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:51.098479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254282+0000","last_change":"2024-04-24T23:12:32.029104+0000","last_active":"2024-04-24T23:12:33.254282+0000","last_peered":"2024-04-24T23:12:33.254282+0000","last_clean":"2024-04-24T23:12:33.254282+0000","last_became_active":"2024-04-24T23:12:32.028704+0000","last_became_peered":"2024-04-24T23:12:32.028704+0000","last_unstale":"2024-04-24T23:12:33.254282+0000","last_undegraded":"2024-04-24T23:12:33.254282+0000","last_fullsized":"2024-04-24T23:12:33.254282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:33.200884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991790+0000","last_change":"2024-04-24T23:12:27.007422+0000","last_active":"2024-04-24T23:12:33.991790+0000","last_peered":"2024-04-24T23:12:33.991790+0000","last_clean":"2024-04-24T23:12:33.991790+0000","last_became_active":"2024-04-24T23:12:27.007024+0000","last_became_peered":"2024-04-24T23:12:27.007024+0000","last_unstale":"2024-04-24T23:12:33.991790+0000","last_undegraded":"2024-04-24T23:12:33.991790+0000","last_fullsized":"2024-04-24T23:12:33.991790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:30.894232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252372+0000","last_change":"2024-04-24T23:12:28.010510+0000","last_active":"2024-04-24T23:12:33.252372+0000","last_peered":"2024-04-24T23:12:33.252372+0000","last_clean":"2024-04-24T23:12:33.252372+0000","last_became_active":"2024-04-24T23:12:28.009244+0000","last_became_peered":"2024-04-24T23:12:28.009244+0000","last_unstale":"2024-04-24T23:12:33.252372+0000","last_undegraded":"2024-04-24T23:12:33.252372+0000","last_fullsized":"2024-04-24T23:12:33.252372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.146563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252026+0000","last_change":"2024-04-24T23:12:32.031156+0000","last_active":"2024-04-24T23:12:33.252026+0000","last_peered":"2024-04-24T23:12:33.252026+0000","last_clean":"2024-04-24T23:12:33.252026+0000","last_became_active":"2024-04-24T23:12:32.030653+0000","last_became_peered":"2024-04-24T23:12:32.030653+0000","last_unstale":"2024-04-24T23:12:33.252026+0000","last_undegraded":"2024-04-24T23:12:33.252026+0000","last_fullsized":"2024-04-24T23:12:33.252026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:48.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993489+0000","last_change":"2024-04-24T23:12:26.998003+0000","last_active":"2024-04-24T23:12:33.993489+0000","last_peered":"2024-04-24T23:12:33.993489+0000","last_clean":"2024-04-24T23:12:33.993489+0000","last_became_active":"2024-04-24T23:12:26.997661+0000","last_became_peered":"2024-04-24T23:12:26.997661+0000","last_unstale":"2024-04-24T23:12:33.993489+0000","last_undegraded":"2024-04-24T23:12:33.993489+0000","last_fullsized":"2024-04-24T23:12:33.993489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:17.995397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992273+0000","last_change":"2024-04-24T23:12:28.004822+0000","last_active":"2024-04-24T23:12:33.992273+0000","last_peered":"2024-04-24T23:12:33.992273+0000","last_clean":"2024-04-24T23:12:33.992273+0000","last_became_active":"2024-04-24T23:12:28.004334+0000","last_became_peered":"2024-04-24T23:12:28.004334+0000","last_unstale":"2024-04-24T23:12:33.992273+0000","last_undegraded":"2024-04-24T23:12:33.992273+0000","last_fullsized":"2024-04-24T23:12:33.992273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.717796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992005+0000","last_change":"2024-04-24T23:12:32.085309+0000","last_active":"2024-04-24T23:12:33.992005+0000","last_peered":"2024-04-24T23:12:33.992005+0000","last_clean":"2024-04-24T23:12:33.992005+0000","last_became_active":"2024-04-24T23:12:32.084892+0000","last_became_peered":"2024-04-24T23:12:32.084892+0000","last_unstale":"2024-04-24T23:12:33.992005+0000","last_undegraded":"2024-04-24T23:12:33.992005+0000","last_fullsized":"2024-04-24T23:12:33.992005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:20.318901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992587+0000","last_change":"2024-04-24T23:12:27.005096+0000","last_active":"2024-04-24T23:12:33.992587+0000","last_peered":"2024-04-24T23:12:33.992587+0000","last_clean":"2024-04-24T23:12:33.992587+0000","last_became_active":"2024-04-24T23:12:27.004699+0000","last_became_peered":"2024-04-24T23:12:27.004699+0000","last_unstale":"2024-04-24T23:12:33.992587+0000","last_undegraded":"2024-04-24T23:12:33.992587+0000","last_fullsized":"2024-04-24T23:12:33.992587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:58.225361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993149+0000","last_change":"2024-04-24T23:12:28.012230+0000","last_active":"2024-04-24T23:12:33.993149+0000","last_peered":"2024-04-24T23:12:33.993149+0000","last_clean":"2024-04-24T23:12:33.993149+0000","last_became_active":"2024-04-24T23:12:28.011900+0000","last_became_peered":"2024-04-24T23:12:28.011900+0000","last_unstale":"2024-04-24T23:12:33.993149+0000","last_undegraded":"2024-04-24T23:12:33.993149+0000","last_fullsized":"2024-04-24T23:12:33.993149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.780548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254018+0000","last_change":"2024-04-24T23:12:32.026388+0000","last_active":"2024-04-24T23:12:33.254018+0000","last_peered":"2024-04-24T23:12:33.254018+0000","last_clean":"2024-04-24T23:12:33.254018+0000","last_became_active":"2024-04-24T23:12:32.025994+0000","last_became_peered":"2024-04-24T23:12:32.025994+0000","last_unstale":"2024-04-24T23:12:33.254018+0000","last_undegraded":"2024-04-24T23:12:33.254018+0000","last_fullsized":"2024-04-24T23:12:33.254018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:13.062781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512525+0000","last_change":"2024-04-24T23:12:27.005080+0000","last_active":"2024-04-24T23:12:33.512525+0000","last_peered":"2024-04-24T23:12:33.512525+0000","last_clean":"2024-04-24T23:12:33.512525+0000","last_became_active":"2024-04-24T23:12:27.004740+0000","last_became_peered":"2024-04-24T23:12:27.004740+0000","last_unstale":"2024-04-24T23:12:33.512525+0000","last_undegraded":"2024-04-24T23:12:33.512525+0000","last_fullsized":"2024-04-24T23:12:33.512525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:22.502399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992913+0000","last_change":"2024-04-24T23:12:28.008016+0000","last_active":"2024-04-24T23:12:33.992913+0000","last_peered":"2024-04-24T23:12:33.992913+0000","last_clean":"2024-04-24T23:12:33.992913+0000","last_became_active":"2024-04-24T23:12:28.007139+0000","last_became_peered":"2024-04-24T23:12:28.007139+0000","last_unstale":"2024-04-24T23:12:33.992913+0000","last_undegraded":"2024-04-24T23:12:33.992913+0000","last_fullsized":"2024-04-24T23:12:33.992913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.698193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254090+0000","last_change":"2024-04-24T23:12:32.085290+0000","last_active":"2024-04-24T23:12:33.254090+0000","last_peered":"2024-04-24T23:12:33.254090+0000","last_clean":"2024-04-24T23:12:33.254090+0000","last_became_active":"2024-04-24T23:12:32.084852+0000","last_became_peered":"2024-04-24T23:12:32.084852+0000","last_unstale":"2024-04-24T23:12:33.254090+0000","last_undegraded":"2024-04-24T23:12:33.254090+0000","last_fullsized":"2024-04-24T23:12:33.254090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.753633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252226+0000","last_change":"2024-04-24T23:12:28.007552+0000","last_active":"2024-04-24T23:12:33.252226+0000","last_peered":"2024-04-24T23:12:33.252226+0000","last_clean":"2024-04-24T23:12:33.252226+0000","last_became_active":"2024-04-24T23:12:28.007196+0000","last_became_peered":"2024-04-24T23:12:28.007196+0000","last_unstale":"2024-04-24T23:12:33.252226+0000","last_undegraded":"2024-04-24T23:12:33.252226+0000","last_fullsized":"2024-04-24T23:12:33.252226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:15.695485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253650+0000","last_change":"2024-04-24T23:12:26.990814+0000","last_active":"2024-04-24T23:12:33.253650+0000","last_peered":"2024-04-24T23:12:33.253650+0000","last_clean":"2024-04-24T23:12:33.253650+0000","last_became_active":"2024-04-24T23:12:26.990476+0000","last_became_peered":"2024-04-24T23:12:26.990476+0000","last_unstale":"2024-04-24T23:12:33.253650+0000","last_undegraded":"2024-04-24T23:12:33.253650+0000","last_fullsized":"2024-04-24T23:12:33.253650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:32.645770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512840+0000","last_change":"2024-04-24T23:12:32.022039+0000","last_active":"2024-04-24T23:12:33.512840+0000","last_peered":"2024-04-24T23:12:33.512840+0000","last_clean":"2024-04-24T23:12:33.512840+0000","last_became_active":"2024-04-24T23:12:32.021514+0000","last_became_peered":"2024-04-24T23:12:32.021514+0000","last_unstale":"2024-04-24T23:12:33.512840+0000","last_undegraded":"2024-04-24T23:12:33.512840+0000","last_fullsized":"2024-04-24T23:12:33.512840+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:42.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993211+0000","last_change":"2024-04-24T23:12:28.008676+0000","last_active":"2024-04-24T23:12:33.993211+0000","last_peered":"2024-04-24T23:12:33.993211+0000","last_clean":"2024-04-24T23:12:33.993211+0000","last_became_active":"2024-04-24T23:12:28.008376+0000","last_became_peered":"2024-04-24T23:12:28.008376+0000","last_unstale":"2024-04-24T23:12:33.993211+0000","last_undegraded":"2024-04-24T23:12:33.993211+0000","last_fullsized":"2024-04-24T23:12:33.993211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:15.358332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991640+0000","last_change":"2024-04-24T23:12:26.994952+0000","last_active":"2024-04-24T23:12:33.991640+0000","last_peered":"2024-04-24T23:12:33.991640+0000","last_clean":"2024-04-24T23:12:33.991640+0000","last_became_active":"2024-04-24T23:12:26.994556+0000","last_became_peered":"2024-04-24T23:12:26.994556+0000","last_unstale":"2024-04-24T23:12:33.991640+0000","last_undegraded":"2024-04-24T23:12:33.991640+0000","last_fullsized":"2024-04-24T23:12:33.991640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.257427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512254+0000","last_change":"2024-04-24T23:11:16.721876+0000","last_active":"2024-04-24T23:12:33.512254+0000","last_peered":"2024-04-24T23:12:33.512254+0000","last_clean":"2024-04-24T23:12:33.512254+0000","last_became_active":"2024-04-24T23:11:16.721291+0000","last_became_peered":"2024-04-24T23:11:16.721291+0000","last_unstale":"2024-04-24T23:12:33.512254+0000","last_undegraded":"2024-04-24T23:12:33.512254+0000","last_fullsized":"2024-04-24T23:12:33.512254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_clean_scrub_stamp":"2024-04-24T23:07:51.257753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.458939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252459+0000","last_change":"2024-04-24T23:12:32.085905+0000","last_active":"2024-04-24T23:12:33.252459+0000","last_peered":"2024-04-24T23:12:33.252459+0000","last_clean":"2024-04-24T23:12:33.252459+0000","last_became_active":"2024-04-24T23:12:32.085374+0000","last_became_peered":"2024-04-24T23:12:32.085374+0000","last_unstale":"2024-04-24T23:12:33.252459+0000","last_undegraded":"2024-04-24T23:12:33.252459+0000","last_fullsized":"2024-04-24T23:12:33.252459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:16.290651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253498+0000","last_change":"2024-04-24T23:12:27.999880+0000","last_active":"2024-04-24T23:12:33.253498+0000","last_peered":"2024-04-24T23:12:33.253498+0000","last_clean":"2024-04-24T23:12:33.253498+0000","last_became_active":"2024-04-24T23:12:27.999543+0000","last_became_peered":"2024-04-24T23:12:27.999543+0000","last_unstale":"2024-04-24T23:12:33.253498+0000","last_undegraded":"2024-04-24T23:12:33.253498+0000","last_fullsized":"2024-04-24T23:12:33.253498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:44.758968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512426+0000","last_change":"2024-04-24T23:12:27.005918+0000","last_active":"2024-04-24T23:12:33.512426+0000","last_peered":"2024-04-24T23:12:33.512426+0000","last_clean":"2024-04-24T23:12:33.512426+0000","last_became_active":"2024-04-24T23:12:27.005522+0000","last_became_peered":"2024-04-24T23:12:27.005522+0000","last_unstale":"2024-04-24T23:12:33.512426+0000","last_undegraded":"2024-04-24T23:12:33.512426+0000","last_fullsized":"2024-04-24T23:12:33.512426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:14.292662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992251+0000","last_change":"2024-04-24T23:12:32.024952+0000","last_active":"2024-04-24T23:12:33.992251+0000","last_peered":"2024-04-24T23:12:33.992251+0000","last_clean":"2024-04-24T23:12:33.992251+0000","last_became_active":"2024-04-24T23:12:32.024404+0000","last_became_peered":"2024-04-24T23:12:32.024404+0000","last_unstale":"2024-04-24T23:12:33.992251+0000","last_undegraded":"2024-04-24T23:12:33.992251+0000","last_fullsized":"2024-04-24T23:12:33.992251+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.377794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993135+0000","last_change":"2024-04-24T23:12:27.007191+0000","last_active":"2024-04-24T23:12:33.993135+0000","last_peered":"2024-04-24T23:12:33.993135+0000","last_clean":"2024-04-24T23:12:33.993135+0000","last_became_active":"2024-04-24T23:12:27.006867+0000","last_became_peered":"2024-04-24T23:12:27.006867+0000","last_unstale":"2024-04-24T23:12:33.993135+0000","last_undegraded":"2024-04-24T23:12:33.993135+0000","last_fullsized":"2024-04-24T23:12:33.993135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:58.112293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254082+0000","last_change":"2024-04-24T23:12:27.996322+0000","last_active":"2024-04-24T23:12:33.254082+0000","last_peered":"2024-04-24T23:12:33.254082+0000","last_clean":"2024-04-24T23:12:33.254082+0000","last_became_active":"2024-04-24T23:12:27.995991+0000","last_became_peered":"2024-04-24T23:12:27.995991+0000","last_unstale":"2024-04-24T23:12:33.254082+0000","last_undegraded":"2024-04-24T23:12:33.254082+0000","last_fullsized":"2024-04-24T23:12:33.254082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:08.960347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991531+0000","last_change":"2024-04-24T23:12:32.084959+0000","last_active":"2024-04-24T23:12:33.991531+0000","last_peered":"2024-04-24T23:12:33.991531+0000","last_clean":"2024-04-24T23:12:33.991531+0000","last_became_active":"2024-04-24T23:12:32.084553+0000","last_became_peered":"2024-04-24T23:12:32.084553+0000","last_unstale":"2024-04-24T23:12:33.991531+0000","last_undegraded":"2024-04-24T23:12:33.991531+0000","last_fullsized":"2024-04-24T23:12:33.991531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.836337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993365+0000","last_change":"2024-04-24T23:12:27.007742+0000","last_active":"2024-04-24T23:12:33.993365+0000","last_peered":"2024-04-24T23:12:33.993365+0000","last_clean":"2024-04-24T23:12:33.993365+0000","last_became_active":"2024-04-24T23:12:27.007423+0000","last_became_peered":"2024-04-24T23:12:27.007423+0000","last_unstale":"2024-04-24T23:12:33.993365+0000","last_undegraded":"2024-04-24T23:12:33.993365+0000","last_fullsized":"2024-04-24T23:12:33.993365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:39.478356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992351+0000","last_change":"2024-04-24T23:12:28.010961+0000","last_active":"2024-04-24T23:12:33.992351+0000","last_peered":"2024-04-24T23:12:33.992351+0000","last_clean":"2024-04-24T23:12:33.992351+0000","last_became_active":"2024-04-24T23:12:28.010641+0000","last_became_peered":"2024-04-24T23:12:28.010641+0000","last_unstale":"2024-04-24T23:12:33.992351+0000","last_undegraded":"2024-04-24T23:12:33.992351+0000","last_fullsized":"2024-04-24T23:12:33.992351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:43.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992679+0000","last_change":"2024-04-24T23:12:32.023687+0000","last_active":"2024-04-24T23:12:33.992679+0000","last_peered":"2024-04-24T23:12:33.992679+0000","last_clean":"2024-04-24T23:12:33.992679+0000","last_became_active":"2024-04-24T23:12:32.023052+0000","last_became_peered":"2024-04-24T23:12:32.023052+0000","last_unstale":"2024-04-24T23:12:33.992679+0000","last_undegraded":"2024-04-24T23:12:33.992679+0000","last_fullsized":"2024-04-24T23:12:33.992679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:39.854030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991977+0000","last_change":"2024-04-24T23:12:27.009743+0000","last_active":"2024-04-24T23:12:33.991977+0000","last_peered":"2024-04-24T23:12:33.991977+0000","last_clean":"2024-04-24T23:12:33.991977+0000","last_became_active":"2024-04-24T23:12:27.009343+0000","last_became_peered":"2024-04-24T23:12:27.009343+0000","last_unstale":"2024-04-24T23:12:33.991977+0000","last_undegraded":"2024-04-24T23:12:33.991977+0000","last_fullsized":"2024-04-24T23:12:33.991977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:20.387738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991705+0000","last_change":"2024-04-24T23:12:28.011895+0000","last_active":"2024-04-24T23:12:33.991705+0000","last_peered":"2024-04-24T23:12:33.991705+0000","last_clean":"2024-04-24T23:12:33.991705+0000","last_became_active":"2024-04-24T23:12:28.011546+0000","last_became_peered":"2024-04-24T23:12:28.011546+0000","last_unstale":"2024-04-24T23:12:33.991705+0000","last_undegraded":"2024-04-24T23:12:33.991705+0000","last_fullsized":"2024-04-24T23:12:33.991705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:17.982200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992180+0000","last_change":"2024-04-24T23:12:32.086209+0000","last_active":"2024-04-24T23:12:33.992180+0000","last_peered":"2024-04-24T23:12:33.992180+0000","last_clean":"2024-04-24T23:12:33.992180+0000","last_became_active":"2024-04-24T23:12:32.085752+0000","last_became_peered":"2024-04-24T23:12:32.085752+0000","last_unstale":"2024-04-24T23:12:33.992180+0000","last_undegraded":"2024-04-24T23:12:33.992180+0000","last_fullsized":"2024-04-24T23:12:33.992180+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:52.120905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:51.886798+0000","last_change":"2024-04-24T23:12:26.996355+0000","last_active":"2024-04-24T23:12:51.886798+0000","last_peered":"2024-04-24T23:12:51.886798+0000","last_clean":"2024-04-24T23:12:51.886798+0000","last_became_active":"2024-04-24T23:12:26.995486+0000","last_became_peered":"2024-04-24T23:12:26.995486+0000","last_unstale":"2024-04-24T23:12:51.886798+0000","last_undegraded":"2024-04-24T23:12:51.886798+0000","last_fullsized":"2024-04-24T23:12:51.886798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:48.981985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991778+0000","last_change":"2024-04-24T23:12:28.004610+0000","last_active":"2024-04-24T23:12:33.991778+0000","last_peered":"2024-04-24T23:12:33.991778+0000","last_clean":"2024-04-24T23:12:33.991778+0000","last_became_active":"2024-04-24T23:12:28.004051+0000","last_became_peered":"2024-04-24T23:12:28.004051+0000","last_unstale":"2024-04-24T23:12:33.991778+0000","last_undegraded":"2024-04-24T23:12:33.991778+0000","last_fullsized":"2024-04-24T23:12:33.991778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:42.656954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992985+0000","last_change":"2024-04-24T23:12:32.029269+0000","last_active":"2024-04-24T23:12:33.992985+0000","last_peered":"2024-04-24T23:12:33.992985+0000","last_clean":"2024-04-24T23:12:33.992985+0000","last_became_active":"2024-04-24T23:12:32.028880+0000","last_became_peered":"2024-04-24T23:12:32.028880+0000","last_unstale":"2024-04-24T23:12:33.992985+0000","last_undegraded":"2024-04-24T23:12:33.992985+0000","last_fullsized":"2024-04-24T23:12:33.992985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:19.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.873415+0000","last_change":"2024-04-24T23:12:27.006357+0000","last_active":"2024-04-24T23:12:53.873415+0000","last_peered":"2024-04-24T23:12:53.873415+0000","last_clean":"2024-04-24T23:12:53.873415+0000","last_became_active":"2024-04-24T23:12:27.006040+0000","last_became_peered":"2024-04-24T23:12:27.006040+0000","last_unstale":"2024-04-24T23:12:53.873415+0000","last_undegraded":"2024-04-24T23:12:53.873415+0000","last_fullsized":"2024-04-24T23:12:53.873415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:27.520663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992003+0000","last_change":"2024-04-24T23:12:28.008909+0000","last_active":"2024-04-24T23:12:33.992003+0000","last_peered":"2024-04-24T23:12:33.992003+0000","last_clean":"2024-04-24T23:12:33.992003+0000","last_became_active":"2024-04-24T23:12:28.008552+0000","last_became_peered":"2024-04-24T23:12:28.008552+0000","last_unstale":"2024-04-24T23:12:33.992003+0000","last_undegraded":"2024-04-24T23:12:33.992003+0000","last_fullsized":"2024-04-24T23:12:33.992003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:20.628234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992614+0000","last_change":"2024-04-24T23:12:32.029720+0000","last_active":"2024-04-24T23:12:33.992614+0000","last_peered":"2024-04-24T23:12:33.992614+0000","last_clean":"2024-04-24T23:12:33.992614+0000","last_became_active":"2024-04-24T23:12:32.029311+0000","last_became_peered":"2024-04-24T23:12:32.029311+0000","last_unstale":"2024-04-24T23:12:33.992614+0000","last_undegraded":"2024-04-24T23:12:33.992614+0000","last_fullsized":"2024-04-24T23:12:33.992614+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:37.832102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992420+0000","last_change":"2024-04-24T23:12:26.995552+0000","last_active":"2024-04-24T23:12:33.992420+0000","last_peered":"2024-04-24T23:12:33.992420+0000","last_clean":"2024-04-24T23:12:33.992420+0000","last_became_active":"2024-04-24T23:12:26.994922+0000","last_became_peered":"2024-04-24T23:12:26.994922+0000","last_unstale":"2024-04-24T23:12:33.992420+0000","last_undegraded":"2024-04-24T23:12:33.992420+0000","last_fullsized":"2024-04-24T23:12:33.992420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:17.981344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992726+0000","last_change":"2024-04-24T23:12:28.008802+0000","last_active":"2024-04-24T23:12:33.992726+0000","last_peered":"2024-04-24T23:12:33.992726+0000","last_clean":"2024-04-24T23:12:33.992726+0000","last_became_active":"2024-04-24T23:12:28.008496+0000","last_became_peered":"2024-04-24T23:12:28.008496+0000","last_unstale":"2024-04-24T23:12:33.992726+0000","last_undegraded":"2024-04-24T23:12:33.992726+0000","last_fullsized":"2024-04-24T23:12:33.992726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:54.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992280+0000","last_change":"2024-04-24T23:12:32.085555+0000","last_active":"2024-04-24T23:12:33.992280+0000","last_peered":"2024-04-24T23:12:33.992280+0000","last_clean":"2024-04-24T23:12:33.992280+0000","last_became_active":"2024-04-24T23:12:32.085085+0000","last_became_peered":"2024-04-24T23:12:32.085085+0000","last_unstale":"2024-04-24T23:12:33.992280+0000","last_undegraded":"2024-04-24T23:12:33.992280+0000","last_fullsized":"2024-04-24T23:12:33.992280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:24.563964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992292+0000","last_change":"2024-04-24T23:12:27.003221+0000","last_active":"2024-04-24T23:12:33.992292+0000","last_peered":"2024-04-24T23:12:33.992292+0000","last_clean":"2024-04-24T23:12:33.992292+0000","last_became_active":"2024-04-24T23:12:27.002879+0000","last_became_peered":"2024-04-24T23:12:27.002879+0000","last_unstale":"2024-04-24T23:12:33.992292+0000","last_undegraded":"2024-04-24T23:12:33.992292+0000","last_fullsized":"2024-04-24T23:12:33.992292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:10.548202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512526+0000","last_change":"2024-04-24T23:12:28.014865+0000","last_active":"2024-04-24T23:12:33.512526+0000","last_peered":"2024-04-24T23:12:33.512526+0000","last_clean":"2024-04-24T23:12:33.512526+0000","last_became_active":"2024-04-24T23:12:28.014538+0000","last_became_peered":"2024-04-24T23:12:28.014538+0000","last_unstale":"2024-04-24T23:12:33.512526+0000","last_undegraded":"2024-04-24T23:12:33.512526+0000","last_fullsized":"2024-04-24T23:12:33.512526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:55.306883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256077+0000","last_change":"2024-04-24T23:12:32.024849+0000","last_active":"2024-04-24T23:12:33.256077+0000","last_peered":"2024-04-24T23:12:33.256077+0000","last_clean":"2024-04-24T23:12:33.256077+0000","last_became_active":"2024-04-24T23:12:32.024412+0000","last_became_peered":"2024-04-24T23:12:32.024412+0000","last_unstale":"2024-04-24T23:12:33.256077+0000","last_undegraded":"2024-04-24T23:12:33.256077+0000","last_fullsized":"2024-04-24T23:12:33.256077+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:16.884565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992908+0000","last_change":"2024-04-24T23:12:27.007284+0000","last_active":"2024-04-24T23:12:33.992908+0000","last_peered":"2024-04-24T23:12:33.992908+0000","last_clean":"2024-04-24T23:12:33.992908+0000","last_became_active":"2024-04-24T23:12:27.006916+0000","last_became_peered":"2024-04-24T23:12:27.006916+0000","last_unstale":"2024-04-24T23:12:33.992908+0000","last_undegraded":"2024-04-24T23:12:33.992908+0000","last_fullsized":"2024-04-24T23:12:33.992908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:40.333387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512970+0000","last_change":"2024-04-24T23:12:28.011603+0000","last_active":"2024-04-24T23:12:33.512970+0000","last_peered":"2024-04-24T23:12:33.512970+0000","last_clean":"2024-04-24T23:12:33.512970+0000","last_became_active":"2024-04-24T23:12:28.011268+0000","last_became_peered":"2024-04-24T23:12:28.011268+0000","last_unstale":"2024-04-24T23:12:33.512970+0000","last_undegraded":"2024-04-24T23:12:33.512970+0000","last_fullsized":"2024-04-24T23:12:33.512970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.693024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991494+0000","last_change":"2024-04-24T23:12:32.033965+0000","last_active":"2024-04-24T23:12:33.991494+0000","last_peered":"2024-04-24T23:12:33.991494+0000","last_clean":"2024-04-24T23:12:33.991494+0000","last_became_active":"2024-04-24T23:12:32.033472+0000","last_became_peered":"2024-04-24T23:12:32.033472+0000","last_unstale":"2024-04-24T23:12:33.991494+0000","last_undegraded":"2024-04-24T23:12:33.991494+0000","last_fullsized":"2024-04-24T23:12:33.991494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:05.243047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252443+0000","last_change":"2024-04-24T23:12:26.991981+0000","last_active":"2024-04-24T23:12:33.252443+0000","last_peered":"2024-04-24T23:12:33.252443+0000","last_clean":"2024-04-24T23:12:33.252443+0000","last_became_active":"2024-04-24T23:12:26.990707+0000","last_became_peered":"2024-04-24T23:12:26.990707+0000","last_unstale":"2024-04-24T23:12:33.252443+0000","last_undegraded":"2024-04-24T23:12:33.252443+0000","last_fullsized":"2024-04-24T23:12:33.252443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:36.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512770+0000","last_change":"2024-04-24T23:12:28.010428+0000","last_active":"2024-04-24T23:12:33.512770+0000","last_peered":"2024-04-24T23:12:33.512770+0000","last_clean":"2024-04-24T23:12:33.512770+0000","last_became_active":"2024-04-24T23:12:28.010096+0000","last_became_peered":"2024-04-24T23:12:28.010096+0000","last_unstale":"2024-04-24T23:12:33.512770+0000","last_undegraded":"2024-04-24T23:12:33.512770+0000","last_fullsized":"2024-04-24T23:12:33.512770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:31.949617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.251812+0000","last_change":"2024-04-24T23:12:32.023828+0000","last_active":"2024-04-24T23:12:33.251812+0000","last_peered":"2024-04-24T23:12:33.251812+0000","last_clean":"2024-04-24T23:12:33.251812+0000","last_became_active":"2024-04-24T23:12:32.023437+0000","last_became_peered":"2024-04-24T23:12:32.023437+0000","last_unstale":"2024-04-24T23:12:33.251812+0000","last_undegraded":"2024-04-24T23:12:33.251812+0000","last_fullsized":"2024-04-24T23:12:33.251812+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.186860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254142+0000","last_change":"2024-04-24T23:12:26.993711+0000","last_active":"2024-04-24T23:12:33.254142+0000","last_peered":"2024-04-24T23:12:33.254142+0000","last_clean":"2024-04-24T23:12:33.254142+0000","last_became_active":"2024-04-24T23:12:26.993368+0000","last_became_peered":"2024-04-24T23:12:26.993368+0000","last_unstale":"2024-04-24T23:12:33.254142+0000","last_undegraded":"2024-04-24T23:12:33.254142+0000","last_fullsized":"2024-04-24T23:12:33.254142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:58.843747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512964+0000","last_change":"2024-04-24T23:12:28.003026+0000","last_active":"2024-04-24T23:12:33.512964+0000","last_peered":"2024-04-24T23:12:33.512964+0000","last_clean":"2024-04-24T23:12:33.512964+0000","last_became_active":"2024-04-24T23:12:28.002693+0000","last_became_peered":"2024-04-24T23:12:28.002693+0000","last_unstale":"2024-04-24T23:12:33.512964+0000","last_undegraded":"2024-04-24T23:12:33.512964+0000","last_fullsized":"2024-04-24T23:12:33.512964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:30.926251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992017+0000","last_change":"2024-04-24T23:12:28.013414+0000","last_active":"2024-04-24T23:12:33.992017+0000","last_peered":"2024-04-24T23:12:33.992017+0000","last_clean":"2024-04-24T23:12:33.992017+0000","last_became_active":"2024-04-24T23:12:28.013030+0000","last_became_peered":"2024-04-24T23:12:28.013030+0000","last_unstale":"2024-04-24T23:12:33.992017+0000","last_undegraded":"2024-04-24T23:12:33.992017+0000","last_fullsized":"2024-04-24T23:12:33.992017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:48.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993146+0000","last_change":"2024-04-24T23:12:28.004821+0000","last_active":"2024-04-24T23:12:33.993146+0000","last_peered":"2024-04-24T23:12:33.993146+0000","last_clean":"2024-04-24T23:12:33.993146+0000","last_became_active":"2024-04-24T23:12:28.003162+0000","last_became_peered":"2024-04-24T23:12:28.003162+0000","last_unstale":"2024-04-24T23:12:33.993146+0000","last_undegraded":"2024-04-24T23:12:33.993146+0000","last_fullsized":"2024-04-24T23:12:33.993146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:59.109665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992356+0000","last_change":"2024-04-24T23:12:28.002115+0000","last_active":"2024-04-24T23:12:33.992356+0000","last_peered":"2024-04-24T23:12:33.992356+0000","last_clean":"2024-04-24T23:12:33.992356+0000","last_became_active":"2024-04-24T23:12:28.001727+0000","last_became_peered":"2024-04-24T23:12:28.001727+0000","last_unstale":"2024-04-24T23:12:33.992356+0000","last_undegraded":"2024-04-24T23:12:33.992356+0000","last_fullsized":"2024-04-24T23:12:33.992356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:14.348843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991577+0000","last_change":"2024-04-24T23:12:28.007784+0000","last_active":"2024-04-24T23:12:33.991577+0000","last_peered":"2024-04-24T23:12:33.991577+0000","last_clean":"2024-04-24T23:12:33.991577+0000","last_became_active":"2024-04-24T23:12:28.007481+0000","last_became_peered":"2024-04-24T23:12:28.007481+0000","last_unstale":"2024-04-24T23:12:33.991577+0000","last_undegraded":"2024-04-24T23:12:33.991577+0000","last_fullsized":"2024-04-24T23:12:33.991577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:33.753448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254000+0000","last_change":"2024-04-24T23:12:28.009100+0000","last_active":"2024-04-24T23:12:33.254000+0000","last_peered":"2024-04-24T23:12:33.254000+0000","last_clean":"2024-04-24T23:12:33.254000+0000","last_became_active":"2024-04-24T23:12:28.008782+0000","last_became_peered":"2024-04-24T23:12:28.008782+0000","last_unstale":"2024-04-24T23:12:33.254000+0000","last_undegraded":"2024-04-24T23:12:33.254000+0000","last_fullsized":"2024-04-24T23:12:33.254000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:44.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992502+0000","last_change":"2024-04-24T23:12:28.008129+0000","last_active":"2024-04-24T23:12:33.992502+0000","last_peered":"2024-04-24T23:12:33.992502+0000","last_clean":"2024-04-24T23:12:33.992502+0000","last_became_active":"2024-04-24T23:12:28.007819+0000","last_became_peered":"2024-04-24T23:12:28.007819+0000","last_unstale":"2024-04-24T23:12:33.992502+0000","last_undegraded":"2024-04-24T23:12:33.992502+0000","last_fullsized":"2024-04-24T23:12:33.992502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.234705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993247+0000","last_change":"2024-04-24T23:12:27.993188+0000","last_active":"2024-04-24T23:12:33.993247+0000","last_peered":"2024-04-24T23:12:33.993247+0000","last_clean":"2024-04-24T23:12:33.993247+0000","last_became_active":"2024-04-24T23:12:27.992832+0000","last_became_peered":"2024-04-24T23:12:27.992832+0000","last_unstale":"2024-04-24T23:12:33.993247+0000","last_undegraded":"2024-04-24T23:12:33.993247+0000","last_fullsized":"2024-04-24T23:12:33.993247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:58.623164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992528+0000","last_change":"2024-04-24T23:12:28.004099+0000","last_active":"2024-04-24T23:12:33.992528+0000","last_peered":"2024-04-24T23:12:33.992528+0000","last_clean":"2024-04-24T23:12:33.992528+0000","last_became_active":"2024-04-24T23:12:28.003802+0000","last_became_peered":"2024-04-24T23:12:28.003802+0000","last_unstale":"2024-04-24T23:12:33.992528+0000","last_undegraded":"2024-04-24T23:12:33.992528+0000","last_fullsized":"2024-04-24T23:12:33.992528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:25.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252100+0000","last_change":"2024-04-24T23:12:28.007700+0000","last_active":"2024-04-24T23:12:33.252100+0000","last_peered":"2024-04-24T23:12:33.252100+0000","last_clean":"2024-04-24T23:12:33.252100+0000","last_became_active":"2024-04-24T23:12:28.002923+0000","last_became_peered":"2024-04-24T23:12:28.002923+0000","last_unstale":"2024-04-24T23:12:33.252100+0000","last_undegraded":"2024-04-24T23:12:33.252100+0000","last_fullsized":"2024-04-24T23:12:33.252100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:46.736096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992156+0000","last_change":"2024-04-24T23:12:28.002194+0000","last_active":"2024-04-24T23:12:33.992156+0000","last_peered":"2024-04-24T23:12:33.992156+0000","last_clean":"2024-04-24T23:12:33.992156+0000","last_became_active":"2024-04-24T23:12:28.001703+0000","last_became_peered":"2024-04-24T23:12:28.001703+0000","last_unstale":"2024-04-24T23:12:33.992156+0000","last_undegraded":"2024-04-24T23:12:33.992156+0000","last_fullsized":"2024-04-24T23:12:33.992156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:20.887025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256368+0000","last_change":"2024-04-24T23:12:28.015980+0000","last_active":"2024-04-24T23:12:33.256368+0000","last_peered":"2024-04-24T23:12:33.256368+0000","last_clean":"2024-04-24T23:12:33.256368+0000","last_became_active":"2024-04-24T23:12:28.013510+0000","last_became_peered":"2024-04-24T23:12:28.013510+0000","last_unstale":"2024-04-24T23:12:33.256368+0000","last_undegraded":"2024-04-24T23:12:33.256368+0000","last_fullsized":"2024-04-24T23:12:33.256368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:37.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253879+0000","last_change":"2024-04-24T23:12:28.001883+0000","last_active":"2024-04-24T23:12:33.253879+0000","last_peered":"2024-04-24T23:12:33.253879+0000","last_clean":"2024-04-24T23:12:33.253879+0000","last_became_active":"2024-04-24T23:12:28.001499+0000","last_became_peered":"2024-04-24T23:12:28.001499+0000","last_unstale":"2024-04-24T23:12:33.253879+0000","last_undegraded":"2024-04-24T23:12:33.253879+0000","last_fullsized":"2024-04-24T23:12:33.253879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:46.805548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992529+0000","last_change":"2024-04-24T23:12:28.004660+0000","last_active":"2024-04-24T23:12:33.992529+0000","last_peered":"2024-04-24T23:12:33.992529+0000","last_clean":"2024-04-24T23:12:33.992529+0000","last_became_active":"2024-04-24T23:12:28.004328+0000","last_became_peered":"2024-04-24T23:12:28.004328+0000","last_unstale":"2024-04-24T23:12:33.992529+0000","last_undegraded":"2024-04-24T23:12:33.992529+0000","last_fullsized":"2024-04-24T23:12:33.992529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:04.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4820,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":17780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":482291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1077497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":482493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037792,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":482313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1593344,"data_stored":1072659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":485551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":483187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":485585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":482117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":481731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1601536,"data_stored":1072259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:12:56.560 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:12:56.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 -- 172.21.15.112:0/699509168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 msgr2=0x7f4264101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 --2- 172.21.15.112:0/699509168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 0x7f4264101d00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4260009900 tx=0x7f426002a410 comp rx=0 tx=0).stop 2024-04-24T23:12:56.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 -- 172.21.15.112:0/699509168 shutdown_connections 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 --2- 172.21.15.112:0/699509168 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4264102240 0x7f4264108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 --2- 172.21.15.112:0/699509168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 0x7f4264101d00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 --2- 172.21.15.112:0/699509168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4264100690 0x7f4264100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 -- 172.21.15.112:0/699509168 >> 172.21.15.112:0/699509168 conn(0x7f42640fbc20 msgr2=0x7f42640fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 -- 172.21.15.112:0/699509168 shutdown_connections 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.689+0000 7f426b464640 1 -- 172.21.15.112:0/699509168 wait complete. 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f426b464640 1 Processor -- start 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f426b464640 1 -- start start 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f426b464640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4264100690 0x7f426406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f426b464640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 0x7f426419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f426b464640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4264102240 0x7f426419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f426b464640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f426410c490 con 0x7f4264101880 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f426b464640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f426410c310 con 0x7f4264100690 2024-04-24T23:12:56.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f426b464640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f426410c610 con 0x7f4264102240 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f42689ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 0x7f426419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f42689ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 0x7f426419c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38010/0 (socket says 172.21.15.112:38010) 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f42689ff640 1 -- 172.21.15.112:0/2403904697 learned_addr learned my addr 172.21.15.112:0/2403904697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f4269200640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4264100690 0x7f426406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f4269a01640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4264102240 0x7f426419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f42689ff640 1 -- 172.21.15.112:0/2403904697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4264102240 msgr2=0x7f426419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f42689ff640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4264102240 0x7f426419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f42689ff640 1 -- 172.21.15.112:0/2403904697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4264100690 msgr2=0x7f426406a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f42689ff640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4264100690 0x7f426406a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f42689ff640 1 -- 172.21.15.112:0/2403904697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426419f770 con 0x7f4264101880 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.690+0000 7f4269a01640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4264102240 0x7f426419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.691+0000 7f42689ff640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 0x7f426419c840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42600316a0 tx=0x7f4260032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.691+0000 7f425a7fc640 1 -- 172.21.15.112:0/2403904697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426003f030 con 0x7f4264101880 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.691+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426419f9a0 con 0x7f4264101880 2024-04-24T23:12:56.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.691+0000 7f425a7fc640 1 -- 172.21.15.112:0/2403904697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4260003d00 con 0x7f4264101880 2024-04-24T23:12:56.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.691+0000 7f425a7fc640 1 -- 172.21.15.112:0/2403904697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260035070 con 0x7f4264101880 2024-04-24T23:12:56.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.691+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426419fe90 con 0x7f4264101880 2024-04-24T23:12:56.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.692+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42641078e0 con 0x7f4264101880 2024-04-24T23:12:56.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.693+0000 7f425a7fc640 1 -- 172.21.15.112:0/2403904697 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4260042030 con 0x7f4264101880 2024-04-24T23:12:56.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.693+0000 7f425a7fc640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f424c0a2c20 0x7f424c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:56.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.694+0000 7f4269200640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f424c0a2c20 0x7f424c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:56.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.694+0000 7f425a7fc640 1 -- 172.21.15.112:0/2403904697 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f42600ef9e0 con 0x7f4264101880 2024-04-24T23:12:56.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.694+0000 7f4269200640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f424c0a2c20 0x7f424c0a50e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f42641016e0 tx=0x7f4254005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:56.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.699+0000 7f425a7fc640 1 -- 172.21.15.112:0/2403904697 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f42600bc5a0 con 0x7f4264101880 2024-04-24T23:12:56.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.872+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4264061b80 con 0x7f424c0a2c20 2024-04-24T23:12:56.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.882+0000 7f425a7fc640 1 -- 172.21.15.112:0/2403904697 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340380 (secure 0 0 0) 0x7f4264061b80 con 0x7f424c0a2c20 2024-04-24T23:12:56.887 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:12:56.889 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f424c0a2c20 msgr2=0x7f424c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f424c0a2c20 0x7f424c0a50e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f42641016e0 tx=0x7f4254005f90 comp rx=0 tx=0).stop 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 msgr2=0x7f426419c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 0x7f426419c840 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f42600316a0 tx=0x7f4260032d70 comp rx=0 tx=0).stop 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 shutdown_connections 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f424c0a2c20 0x7f424c0a50e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4264102240 0x7f426419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4264101880 0x7f426419c840 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 --2- 172.21.15.112:0/2403904697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4264100690 0x7f426406a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.885+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 >> 172.21.15.112:0/2403904697 conn(0x7f42640fbc20 msgr2=0x7f4264104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:56.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.886+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 shutdown_connections 2024-04-24T23:12:56.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:56.886+0000 7f426b464640 1 -- 172.21.15.112:0/2403904697 wait complete. 2024-04-24T23:12:56.900 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-04-24T23:12:56.789903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600938,"num_objects":85,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":149,"num_read_kb":128,"num_write":353,"num_write_kb":2998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":292,"ondisk_log_size":292,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379524,"kb_used_data":13732,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562300,"statfs":{"total":1151940427776,"available":1151551795200,"internally_reserved":0,"allocated":14061568,"data_stored":7610262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5930,"num_objects":61,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":123,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993643+0000","last_change":"2024-04-24T23:12:28.013506+0000","last_active":"2024-04-24T23:12:33.993643+0000","last_peered":"2024-04-24T23:12:33.993643+0000","last_clean":"2024-04-24T23:12:33.993643+0000","last_became_active":"2024-04-24T23:12:28.012652+0000","last_became_peered":"2024-04-24T23:12:28.012652+0000","last_unstale":"2024-04-24T23:12:33.993643+0000","last_undegraded":"2024-04-24T23:12:33.993643+0000","last_fullsized":"2024-04-24T23:12:33.993643+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:57.420465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991651+0000","last_change":"2024-04-24T23:12:28.004998+0000","last_active":"2024-04-24T23:12:33.991651+0000","last_peered":"2024-04-24T23:12:33.991651+0000","last_clean":"2024-04-24T23:12:33.991651+0000","last_became_active":"2024-04-24T23:12:28.004706+0000","last_became_peered":"2024-04-24T23:12:28.004706+0000","last_unstale":"2024-04-24T23:12:33.991651+0000","last_undegraded":"2024-04-24T23:12:33.991651+0000","last_fullsized":"2024-04-24T23:12:33.991651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.759515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991865+0000","last_change":"2024-04-24T23:12:28.019170+0000","last_active":"2024-04-24T23:12:33.991865+0000","last_peered":"2024-04-24T23:12:33.991865+0000","last_clean":"2024-04-24T23:12:33.991865+0000","last_became_active":"2024-04-24T23:12:28.018815+0000","last_became_peered":"2024-04-24T23:12:28.018815+0000","last_unstale":"2024-04-24T23:12:33.991865+0000","last_undegraded":"2024-04-24T23:12:33.991865+0000","last_fullsized":"2024-04-24T23:12:33.991865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:43.830598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992683+0000","last_change":"2024-04-24T23:12:28.008662+0000","last_active":"2024-04-24T23:12:33.992683+0000","last_peered":"2024-04-24T23:12:33.992683+0000","last_clean":"2024-04-24T23:12:33.992683+0000","last_became_active":"2024-04-24T23:12:28.008282+0000","last_became_peered":"2024-04-24T23:12:28.008282+0000","last_unstale":"2024-04-24T23:12:33.992683+0000","last_undegraded":"2024-04-24T23:12:33.992683+0000","last_fullsized":"2024-04-24T23:12:33.992683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:17.558899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992177+0000","last_change":"2024-04-24T23:12:28.005212+0000","last_active":"2024-04-24T23:12:33.992177+0000","last_peered":"2024-04-24T23:12:33.992177+0000","last_clean":"2024-04-24T23:12:33.992177+0000","last_became_active":"2024-04-24T23:12:28.004853+0000","last_became_peered":"2024-04-24T23:12:28.004853+0000","last_unstale":"2024-04-24T23:12:33.992177+0000","last_undegraded":"2024-04-24T23:12:33.992177+0000","last_fullsized":"2024-04-24T23:12:33.992177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:33.650922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992649+0000","last_change":"2024-04-24T23:12:28.004546+0000","last_active":"2024-04-24T23:12:33.992649+0000","last_peered":"2024-04-24T23:12:33.992649+0000","last_clean":"2024-04-24T23:12:33.992649+0000","last_became_active":"2024-04-24T23:12:28.004194+0000","last_became_peered":"2024-04-24T23:12:28.004194+0000","last_unstale":"2024-04-24T23:12:33.992649+0000","last_undegraded":"2024-04-24T23:12:33.992649+0000","last_fullsized":"2024-04-24T23:12:33.992649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:57.772271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512359+0000","last_change":"2024-04-24T23:12:28.009436+0000","last_active":"2024-04-24T23:12:33.512359+0000","last_peered":"2024-04-24T23:12:33.512359+0000","last_clean":"2024-04-24T23:12:33.512359+0000","last_became_active":"2024-04-24T23:12:28.008219+0000","last_became_peered":"2024-04-24T23:12:28.008219+0000","last_unstale":"2024-04-24T23:12:33.512359+0000","last_undegraded":"2024-04-24T23:12:33.512359+0000","last_fullsized":"2024-04-24T23:12:33.512359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991331+0000","last_change":"2024-04-24T23:12:28.010749+0000","last_active":"2024-04-24T23:12:33.991331+0000","last_peered":"2024-04-24T23:12:33.991331+0000","last_clean":"2024-04-24T23:12:33.991331+0000","last_became_active":"2024-04-24T23:12:28.008999+0000","last_became_peered":"2024-04-24T23:12:28.008999+0000","last_unstale":"2024-04-24T23:12:33.991331+0000","last_undegraded":"2024-04-24T23:12:33.991331+0000","last_fullsized":"2024-04-24T23:12:33.991331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:24:23.327427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256720+0000","last_change":"2024-04-24T23:12:28.004948+0000","last_active":"2024-04-24T23:12:33.256720+0000","last_peered":"2024-04-24T23:12:33.256720+0000","last_clean":"2024-04-24T23:12:33.256720+0000","last_became_active":"2024-04-24T23:12:28.004380+0000","last_became_peered":"2024-04-24T23:12:28.004380+0000","last_unstale":"2024-04-24T23:12:33.256720+0000","last_undegraded":"2024-04-24T23:12:33.256720+0000","last_fullsized":"2024-04-24T23:12:33.256720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:53.999210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993173+0000","last_change":"2024-04-24T23:12:28.005192+0000","last_active":"2024-04-24T23:12:33.993173+0000","last_peered":"2024-04-24T23:12:33.993173+0000","last_clean":"2024-04-24T23:12:33.993173+0000","last_became_active":"2024-04-24T23:12:28.004860+0000","last_became_peered":"2024-04-24T23:12:28.004860+0000","last_unstale":"2024-04-24T23:12:33.993173+0000","last_undegraded":"2024-04-24T23:12:33.993173+0000","last_fullsized":"2024-04-24T23:12:33.993173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:21.874408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512846+0000","last_change":"2024-04-24T23:12:28.016098+0000","last_active":"2024-04-24T23:12:33.512846+0000","last_peered":"2024-04-24T23:12:33.512846+0000","last_clean":"2024-04-24T23:12:33.512846+0000","last_became_active":"2024-04-24T23:12:28.015736+0000","last_became_peered":"2024-04-24T23:12:28.015736+0000","last_unstale":"2024-04-24T23:12:33.512846+0000","last_undegraded":"2024-04-24T23:12:33.512846+0000","last_fullsized":"2024-04-24T23:12:33.512846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.071797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992553+0000","last_change":"2024-04-24T23:12:28.010651+0000","last_active":"2024-04-24T23:12:33.992553+0000","last_peered":"2024-04-24T23:12:33.992553+0000","last_clean":"2024-04-24T23:12:33.992553+0000","last_became_active":"2024-04-24T23:12:28.009700+0000","last_became_peered":"2024-04-24T23:12:28.009700+0000","last_unstale":"2024-04-24T23:12:33.992553+0000","last_undegraded":"2024-04-24T23:12:33.992553+0000","last_fullsized":"2024-04-24T23:12:33.992553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:26.172386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253853+0000","last_change":"2024-04-24T23:12:28.005743+0000","last_active":"2024-04-24T23:12:33.253853+0000","last_peered":"2024-04-24T23:12:33.253853+0000","last_clean":"2024-04-24T23:12:33.253853+0000","last_became_active":"2024-04-24T23:12:28.005450+0000","last_became_peered":"2024-04-24T23:12:28.005450+0000","last_unstale":"2024-04-24T23:12:33.253853+0000","last_undegraded":"2024-04-24T23:12:33.253853+0000","last_fullsized":"2024-04-24T23:12:33.253853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:14.858212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991308+0000","last_change":"2024-04-24T23:12:28.004140+0000","last_active":"2024-04-24T23:12:33.991308+0000","last_peered":"2024-04-24T23:12:33.991308+0000","last_clean":"2024-04-24T23:12:33.991308+0000","last_became_active":"2024-04-24T23:12:28.003794+0000","last_became_peered":"2024-04-24T23:12:28.003794+0000","last_unstale":"2024-04-24T23:12:33.991308+0000","last_undegraded":"2024-04-24T23:12:33.991308+0000","last_fullsized":"2024-04-24T23:12:33.991308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:02.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991361+0000","last_change":"2024-04-24T23:12:28.006834+0000","last_active":"2024-04-24T23:12:33.991361+0000","last_peered":"2024-04-24T23:12:33.991361+0000","last_clean":"2024-04-24T23:12:33.991361+0000","last_became_active":"2024-04-24T23:12:28.006504+0000","last_became_peered":"2024-04-24T23:12:28.006504+0000","last_unstale":"2024-04-24T23:12:33.991361+0000","last_undegraded":"2024-04-24T23:12:33.991361+0000","last_fullsized":"2024-04-24T23:12:33.991361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:21.194464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256036+0000","last_change":"2024-04-24T23:12:28.013132+0000","last_active":"2024-04-24T23:12:33.256036+0000","last_peered":"2024-04-24T23:12:33.256036+0000","last_clean":"2024-04-24T23:12:33.256036+0000","last_became_active":"2024-04-24T23:12:28.012751+0000","last_became_peered":"2024-04-24T23:12:28.012751+0000","last_unstale":"2024-04-24T23:12:33.256036+0000","last_undegraded":"2024-04-24T23:12:33.256036+0000","last_fullsized":"2024-04-24T23:12:33.256036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.942985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993478+0000","last_change":"2024-04-24T23:12:28.013719+0000","last_active":"2024-04-24T23:12:33.993478+0000","last_peered":"2024-04-24T23:12:33.993478+0000","last_clean":"2024-04-24T23:12:33.993478+0000","last_became_active":"2024-04-24T23:12:28.013108+0000","last_became_peered":"2024-04-24T23:12:28.013108+0000","last_unstale":"2024-04-24T23:12:33.993478+0000","last_undegraded":"2024-04-24T23:12:33.993478+0000","last_fullsized":"2024-04-24T23:12:33.993478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:38.248867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993476+0000","last_change":"2024-04-24T23:12:28.012386+0000","last_active":"2024-04-24T23:12:33.993476+0000","last_peered":"2024-04-24T23:12:33.993476+0000","last_clean":"2024-04-24T23:12:33.993476+0000","last_became_active":"2024-04-24T23:12:28.011340+0000","last_became_peered":"2024-04-24T23:12:28.011340+0000","last_unstale":"2024-04-24T23:12:33.993476+0000","last_undegraded":"2024-04-24T23:12:33.993476+0000","last_fullsized":"2024-04-24T23:12:33.993476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:48.115600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993007+0000","last_change":"2024-04-24T23:12:28.004033+0000","last_active":"2024-04-24T23:12:33.993007+0000","last_peered":"2024-04-24T23:12:33.993007+0000","last_clean":"2024-04-24T23:12:33.993007+0000","last_became_active":"2024-04-24T23:12:28.003648+0000","last_became_peered":"2024-04-24T23:12:28.003648+0000","last_unstale":"2024-04-24T23:12:33.993007+0000","last_undegraded":"2024-04-24T23:12:33.993007+0000","last_fullsized":"2024-04-24T23:12:33.993007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:13.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993074+0000","last_change":"2024-04-24T23:12:32.026590+0000","last_active":"2024-04-24T23:12:33.993074+0000","last_peered":"2024-04-24T23:12:33.993074+0000","last_clean":"2024-04-24T23:12:33.993074+0000","last_became_active":"2024-04-24T23:12:32.026098+0000","last_became_peered":"2024-04-24T23:12:32.026098+0000","last_unstale":"2024-04-24T23:12:33.993074+0000","last_undegraded":"2024-04-24T23:12:33.993074+0000","last_fullsized":"2024-04-24T23:12:33.993074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:55.812641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993122+0000","last_change":"2024-04-24T23:12:27.005325+0000","last_active":"2024-04-24T23:12:33.993122+0000","last_peered":"2024-04-24T23:12:33.993122+0000","last_clean":"2024-04-24T23:12:33.993122+0000","last_became_active":"2024-04-24T23:12:27.004627+0000","last_became_peered":"2024-04-24T23:12:27.004627+0000","last_unstale":"2024-04-24T23:12:33.993122+0000","last_undegraded":"2024-04-24T23:12:33.993122+0000","last_fullsized":"2024-04-24T23:12:33.993122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.985647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992318+0000","last_change":"2024-04-24T23:12:28.007872+0000","last_active":"2024-04-24T23:12:33.992318+0000","last_peered":"2024-04-24T23:12:33.992318+0000","last_clean":"2024-04-24T23:12:33.992318+0000","last_became_active":"2024-04-24T23:12:28.007493+0000","last_became_peered":"2024-04-24T23:12:28.007493+0000","last_unstale":"2024-04-24T23:12:33.992318+0000","last_undegraded":"2024-04-24T23:12:33.992318+0000","last_fullsized":"2024-04-24T23:12:33.992318+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:26.686831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991618+0000","last_change":"2024-04-24T23:12:32.031138+0000","last_active":"2024-04-24T23:12:33.991618+0000","last_peered":"2024-04-24T23:12:33.991618+0000","last_clean":"2024-04-24T23:12:33.991618+0000","last_became_active":"2024-04-24T23:12:32.030626+0000","last_became_peered":"2024-04-24T23:12:32.030626+0000","last_unstale":"2024-04-24T23:12:33.991618+0000","last_undegraded":"2024-04-24T23:12:33.991618+0000","last_fullsized":"2024-04-24T23:12:33.991618+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992668+0000","last_change":"2024-04-24T23:12:26.999954+0000","last_active":"2024-04-24T23:12:33.992668+0000","last_peered":"2024-04-24T23:12:33.992668+0000","last_clean":"2024-04-24T23:12:33.992668+0000","last_became_active":"2024-04-24T23:12:26.999507+0000","last_became_peered":"2024-04-24T23:12:26.999507+0000","last_unstale":"2024-04-24T23:12:33.992668+0000","last_undegraded":"2024-04-24T23:12:33.992668+0000","last_fullsized":"2024-04-24T23:12:33.992668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:23.984002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991739+0000","last_change":"2024-04-24T23:12:28.019342+0000","last_active":"2024-04-24T23:12:33.991739+0000","last_peered":"2024-04-24T23:12:33.991739+0000","last_clean":"2024-04-24T23:12:33.991739+0000","last_became_active":"2024-04-24T23:12:28.018976+0000","last_became_peered":"2024-04-24T23:12:28.018976+0000","last_unstale":"2024-04-24T23:12:33.991739+0000","last_undegraded":"2024-04-24T23:12:33.991739+0000","last_fullsized":"2024-04-24T23:12:33.991739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:18.737495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992389+0000","last_change":"2024-04-24T23:12:32.031015+0000","last_active":"2024-04-24T23:12:33.992389+0000","last_peered":"2024-04-24T23:12:33.992389+0000","last_clean":"2024-04-24T23:12:33.992389+0000","last_became_active":"2024-04-24T23:12:32.030564+0000","last_became_peered":"2024-04-24T23:12:32.030564+0000","last_unstale":"2024-04-24T23:12:33.992389+0000","last_undegraded":"2024-04-24T23:12:33.992389+0000","last_fullsized":"2024-04-24T23:12:33.992389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:12.247397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'11","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:54.883157+0000","last_change":"2024-04-24T23:12:27.001229+0000","last_active":"2024-04-24T23:12:54.883157+0000","last_peered":"2024-04-24T23:12:54.883157+0000","last_clean":"2024-04-24T23:12:54.883157+0000","last_became_active":"2024-04-24T23:12:27.000889+0000","last_became_peered":"2024-04-24T23:12:27.000889+0000","last_unstale":"2024-04-24T23:12:54.883157+0000","last_undegraded":"2024-04-24T23:12:54.883157+0000","last_fullsized":"2024-04-24T23:12:54.883157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:39.189828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256356+0000","last_change":"2024-04-24T23:12:28.012309+0000","last_active":"2024-04-24T23:12:33.256356+0000","last_peered":"2024-04-24T23:12:33.256356+0000","last_clean":"2024-04-24T23:12:33.256356+0000","last_became_active":"2024-04-24T23:12:28.011537+0000","last_became_peered":"2024-04-24T23:12:28.011537+0000","last_unstale":"2024-04-24T23:12:33.256356+0000","last_undegraded":"2024-04-24T23:12:33.256356+0000","last_fullsized":"2024-04-24T23:12:33.256356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:38.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991619+0000","last_change":"2024-04-24T23:12:32.030830+0000","last_active":"2024-04-24T23:12:33.991619+0000","last_peered":"2024-04-24T23:12:33.991619+0000","last_clean":"2024-04-24T23:12:33.991619+0000","last_became_active":"2024-04-24T23:12:32.030318+0000","last_became_peered":"2024-04-24T23:12:32.030318+0000","last_unstale":"2024-04-24T23:12:33.991619+0000","last_undegraded":"2024-04-24T23:12:33.991619+0000","last_fullsized":"2024-04-24T23:12:33.991619+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:25.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:54.883631+0000","last_change":"2024-04-24T23:12:26.997926+0000","last_active":"2024-04-24T23:12:54.883631+0000","last_peered":"2024-04-24T23:12:54.883631+0000","last_clean":"2024-04-24T23:12:54.883631+0000","last_became_active":"2024-04-24T23:12:26.997552+0000","last_became_peered":"2024-04-24T23:12:26.997552+0000","last_unstale":"2024-04-24T23:12:54.883631+0000","last_undegraded":"2024-04-24T23:12:54.883631+0000","last_fullsized":"2024-04-24T23:12:54.883631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:50.926624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252316+0000","last_change":"2024-04-24T23:12:28.008486+0000","last_active":"2024-04-24T23:12:33.252316+0000","last_peered":"2024-04-24T23:12:33.252316+0000","last_clean":"2024-04-24T23:12:33.252316+0000","last_became_active":"2024-04-24T23:12:28.008150+0000","last_became_peered":"2024-04-24T23:12:28.008150+0000","last_unstale":"2024-04-24T23:12:33.252316+0000","last_undegraded":"2024-04-24T23:12:33.252316+0000","last_fullsized":"2024-04-24T23:12:33.252316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:15.935567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992845+0000","last_change":"2024-04-24T23:12:32.021601+0000","last_active":"2024-04-24T23:12:33.992845+0000","last_peered":"2024-04-24T23:12:33.992845+0000","last_clean":"2024-04-24T23:12:33.992845+0000","last_became_active":"2024-04-24T23:12:32.021141+0000","last_became_peered":"2024-04-24T23:12:32.021141+0000","last_unstale":"2024-04-24T23:12:33.992845+0000","last_undegraded":"2024-04-24T23:12:33.992845+0000","last_fullsized":"2024-04-24T23:12:33.992845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:53.795337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:54.871760+0000","last_change":"2024-04-24T23:12:27.002639+0000","last_active":"2024-04-24T23:12:54.871760+0000","last_peered":"2024-04-24T23:12:54.871760+0000","last_clean":"2024-04-24T23:12:54.871760+0000","last_became_active":"2024-04-24T23:12:27.002295+0000","last_became_peered":"2024-04-24T23:12:27.002295+0000","last_unstale":"2024-04-24T23:12:54.871760+0000","last_undegraded":"2024-04-24T23:12:54.871760+0000","last_fullsized":"2024-04-24T23:12:54.871760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:40.962878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992430+0000","last_change":"2024-04-24T23:12:28.000123+0000","last_active":"2024-04-24T23:12:33.992430+0000","last_peered":"2024-04-24T23:12:33.992430+0000","last_clean":"2024-04-24T23:12:33.992430+0000","last_became_active":"2024-04-24T23:12:27.999778+0000","last_became_peered":"2024-04-24T23:12:27.999778+0000","last_unstale":"2024-04-24T23:12:33.992430+0000","last_undegraded":"2024-04-24T23:12:33.992430+0000","last_fullsized":"2024-04-24T23:12:33.992430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:03.827336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992635+0000","last_change":"2024-04-24T23:12:32.023886+0000","last_active":"2024-04-24T23:12:33.992635+0000","last_peered":"2024-04-24T23:12:33.992635+0000","last_clean":"2024-04-24T23:12:33.992635+0000","last_became_active":"2024-04-24T23:12:32.022796+0000","last_became_peered":"2024-04-24T23:12:32.022796+0000","last_unstale":"2024-04-24T23:12:33.992635+0000","last_undegraded":"2024-04-24T23:12:33.992635+0000","last_fullsized":"2024-04-24T23:12:33.992635+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:17.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:52.899756+0000","last_change":"2024-04-24T23:12:26.999210+0000","last_active":"2024-04-24T23:12:52.899756+0000","last_peered":"2024-04-24T23:12:52.899756+0000","last_clean":"2024-04-24T23:12:52.899756+0000","last_became_active":"2024-04-24T23:12:26.998642+0000","last_became_peered":"2024-04-24T23:12:26.998642+0000","last_unstale":"2024-04-24T23:12:52.899756+0000","last_undegraded":"2024-04-24T23:12:52.899756+0000","last_fullsized":"2024-04-24T23:12:52.899756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:39.171955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252347+0000","last_change":"2024-04-24T23:12:28.006903+0000","last_active":"2024-04-24T23:12:33.252347+0000","last_peered":"2024-04-24T23:12:33.252347+0000","last_clean":"2024-04-24T23:12:33.252347+0000","last_became_active":"2024-04-24T23:12:28.006541+0000","last_became_peered":"2024-04-24T23:12:28.006541+0000","last_unstale":"2024-04-24T23:12:33.252347+0000","last_undegraded":"2024-04-24T23:12:33.252347+0000","last_fullsized":"2024-04-24T23:12:33.252347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:04.739082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993465+0000","last_change":"2024-04-24T23:12:32.028192+0000","last_active":"2024-04-24T23:12:33.993465+0000","last_peered":"2024-04-24T23:12:33.993465+0000","last_clean":"2024-04-24T23:12:33.993465+0000","last_became_active":"2024-04-24T23:12:32.027715+0000","last_became_peered":"2024-04-24T23:12:32.027715+0000","last_unstale":"2024-04-24T23:12:33.993465+0000","last_undegraded":"2024-04-24T23:12:33.993465+0000","last_fullsized":"2024-04-24T23:12:33.993465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:22.577980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:52.899277+0000","last_change":"2024-04-24T23:12:27.000212+0000","last_active":"2024-04-24T23:12:52.899277+0000","last_peered":"2024-04-24T23:12:52.899277+0000","last_clean":"2024-04-24T23:12:52.899277+0000","last_became_active":"2024-04-24T23:12:26.999844+0000","last_became_peered":"2024-04-24T23:12:26.999844+0000","last_unstale":"2024-04-24T23:12:52.899277+0000","last_undegraded":"2024-04-24T23:12:52.899277+0000","last_fullsized":"2024-04-24T23:12:52.899277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:00.837670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991766+0000","last_change":"2024-04-24T23:12:28.002875+0000","last_active":"2024-04-24T23:12:33.991766+0000","last_peered":"2024-04-24T23:12:33.991766+0000","last_clean":"2024-04-24T23:12:33.991766+0000","last_became_active":"2024-04-24T23:12:28.002572+0000","last_became_peered":"2024-04-24T23:12:28.002572+0000","last_unstale":"2024-04-24T23:12:33.991766+0000","last_undegraded":"2024-04-24T23:12:33.991766+0000","last_fullsized":"2024-04-24T23:12:33.991766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:55.931371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991540+0000","last_change":"2024-04-24T23:12:32.024729+0000","last_active":"2024-04-24T23:12:33.991540+0000","last_peered":"2024-04-24T23:12:33.991540+0000","last_clean":"2024-04-24T23:12:33.991540+0000","last_became_active":"2024-04-24T23:12:32.024316+0000","last_became_peered":"2024-04-24T23:12:32.024316+0000","last_unstale":"2024-04-24T23:12:33.991540+0000","last_undegraded":"2024-04-24T23:12:33.991540+0000","last_fullsized":"2024-04-24T23:12:33.991540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:21.668782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.873585+0000","last_change":"2024-04-24T23:12:27.003435+0000","last_active":"2024-04-24T23:12:53.873585+0000","last_peered":"2024-04-24T23:12:53.873585+0000","last_clean":"2024-04-24T23:12:53.873585+0000","last_became_active":"2024-04-24T23:12:27.003101+0000","last_became_peered":"2024-04-24T23:12:27.003101+0000","last_unstale":"2024-04-24T23:12:53.873585+0000","last_undegraded":"2024-04-24T23:12:53.873585+0000","last_fullsized":"2024-04-24T23:12:53.873585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:42.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992833+0000","last_change":"2024-04-24T23:12:28.005056+0000","last_active":"2024-04-24T23:12:33.992833+0000","last_peered":"2024-04-24T23:12:33.992833+0000","last_clean":"2024-04-24T23:12:33.992833+0000","last_became_active":"2024-04-24T23:12:28.004725+0000","last_became_peered":"2024-04-24T23:12:28.004725+0000","last_unstale":"2024-04-24T23:12:33.992833+0000","last_undegraded":"2024-04-24T23:12:33.992833+0000","last_fullsized":"2024-04-24T23:12:33.992833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:49.384950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992089+0000","last_change":"2024-04-24T23:12:32.032947+0000","last_active":"2024-04-24T23:12:33.992089+0000","last_peered":"2024-04-24T23:12:33.992089+0000","last_clean":"2024-04-24T23:12:33.992089+0000","last_became_active":"2024-04-24T23:12:32.032560+0000","last_became_peered":"2024-04-24T23:12:32.032560+0000","last_unstale":"2024-04-24T23:12:33.992089+0000","last_undegraded":"2024-04-24T23:12:33.992089+0000","last_fullsized":"2024-04-24T23:12:33.992089+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:29.282938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:51.888355+0000","last_change":"2024-04-24T23:12:27.000011+0000","last_active":"2024-04-24T23:12:51.888355+0000","last_peered":"2024-04-24T23:12:51.888355+0000","last_clean":"2024-04-24T23:12:51.888355+0000","last_became_active":"2024-04-24T23:12:26.999572+0000","last_became_peered":"2024-04-24T23:12:26.999572+0000","last_unstale":"2024-04-24T23:12:51.888355+0000","last_undegraded":"2024-04-24T23:12:51.888355+0000","last_fullsized":"2024-04-24T23:12:51.888355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:50.453964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256711+0000","last_change":"2024-04-24T23:12:28.012808+0000","last_active":"2024-04-24T23:12:33.256711+0000","last_peered":"2024-04-24T23:12:33.256711+0000","last_clean":"2024-04-24T23:12:33.256711+0000","last_became_active":"2024-04-24T23:12:28.012406+0000","last_became_peered":"2024-04-24T23:12:28.012406+0000","last_unstale":"2024-04-24T23:12:33.256711+0000","last_undegraded":"2024-04-24T23:12:33.256711+0000","last_fullsized":"2024-04-24T23:12:33.256711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:19.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991826+0000","last_change":"2024-04-24T23:12:32.027156+0000","last_active":"2024-04-24T23:12:33.991826+0000","last_peered":"2024-04-24T23:12:33.991826+0000","last_clean":"2024-04-24T23:12:33.991826+0000","last_became_active":"2024-04-24T23:12:32.026743+0000","last_became_peered":"2024-04-24T23:12:32.026743+0000","last_unstale":"2024-04-24T23:12:33.991826+0000","last_undegraded":"2024-04-24T23:12:33.991826+0000","last_fullsized":"2024-04-24T23:12:33.991826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:43.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:54.882863+0000","last_change":"2024-04-24T23:12:27.001832+0000","last_active":"2024-04-24T23:12:54.882863+0000","last_peered":"2024-04-24T23:12:54.882863+0000","last_clean":"2024-04-24T23:12:54.882863+0000","last_became_active":"2024-04-24T23:12:27.001482+0000","last_became_peered":"2024-04-24T23:12:27.001482+0000","last_unstale":"2024-04-24T23:12:54.882863+0000","last_undegraded":"2024-04-24T23:12:54.882863+0000","last_fullsized":"2024-04-24T23:12:54.882863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:28.949767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991854+0000","last_change":"2024-04-24T23:12:28.010507+0000","last_active":"2024-04-24T23:12:33.991854+0000","last_peered":"2024-04-24T23:12:33.991854+0000","last_clean":"2024-04-24T23:12:33.991854+0000","last_became_active":"2024-04-24T23:12:28.010181+0000","last_became_peered":"2024-04-24T23:12:28.010181+0000","last_unstale":"2024-04-24T23:12:33.991854+0000","last_undegraded":"2024-04-24T23:12:33.991854+0000","last_fullsized":"2024-04-24T23:12:33.991854+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:12.624880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991941+0000","last_change":"2024-04-24T23:12:32.023007+0000","last_active":"2024-04-24T23:12:33.991941+0000","last_peered":"2024-04-24T23:12:33.991941+0000","last_clean":"2024-04-24T23:12:33.991941+0000","last_became_active":"2024-04-24T23:12:32.021324+0000","last_became_peered":"2024-04-24T23:12:32.021324+0000","last_unstale":"2024-04-24T23:12:33.991941+0000","last_undegraded":"2024-04-24T23:12:33.991941+0000","last_fullsized":"2024-04-24T23:12:33.991941+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:41.542778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.869336+0000","last_change":"2024-04-24T23:12:26.990562+0000","last_active":"2024-04-24T23:12:53.869336+0000","last_peered":"2024-04-24T23:12:53.869336+0000","last_clean":"2024-04-24T23:12:53.869336+0000","last_became_active":"2024-04-24T23:12:26.990146+0000","last_became_peered":"2024-04-24T23:12:26.990146+0000","last_unstale":"2024-04-24T23:12:53.869336+0000","last_undegraded":"2024-04-24T23:12:53.869336+0000","last_fullsized":"2024-04-24T23:12:53.869336+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.621051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256491+0000","last_change":"2024-04-24T23:12:28.000954+0000","last_active":"2024-04-24T23:12:33.256491+0000","last_peered":"2024-04-24T23:12:33.256491+0000","last_clean":"2024-04-24T23:12:33.256491+0000","last_became_active":"2024-04-24T23:12:28.000610+0000","last_became_peered":"2024-04-24T23:12:28.000610+0000","last_unstale":"2024-04-24T23:12:33.256491+0000","last_undegraded":"2024-04-24T23:12:33.256491+0000","last_fullsized":"2024-04-24T23:12:33.256491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:32.585919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993576+0000","last_change":"2024-04-24T23:12:32.030373+0000","last_active":"2024-04-24T23:12:33.993576+0000","last_peered":"2024-04-24T23:12:33.993576+0000","last_clean":"2024-04-24T23:12:33.993576+0000","last_became_active":"2024-04-24T23:12:32.029954+0000","last_became_peered":"2024-04-24T23:12:32.029954+0000","last_unstale":"2024-04-24T23:12:33.993576+0000","last_undegraded":"2024-04-24T23:12:33.993576+0000","last_fullsized":"2024-04-24T23:12:33.993576+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:28.061509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:51.888657+0000","last_change":"2024-04-24T23:12:26.996719+0000","last_active":"2024-04-24T23:12:51.888657+0000","last_peered":"2024-04-24T23:12:51.888657+0000","last_clean":"2024-04-24T23:12:51.888657+0000","last_became_active":"2024-04-24T23:12:26.996360+0000","last_became_peered":"2024-04-24T23:12:26.996360+0000","last_unstale":"2024-04-24T23:12:51.888657+0000","last_undegraded":"2024-04-24T23:12:51.888657+0000","last_fullsized":"2024-04-24T23:12:51.888657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.526902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992883+0000","last_change":"2024-04-24T23:12:28.002889+0000","last_active":"2024-04-24T23:12:33.992883+0000","last_peered":"2024-04-24T23:12:33.992883+0000","last_clean":"2024-04-24T23:12:33.992883+0000","last_became_active":"2024-04-24T23:12:28.002540+0000","last_became_peered":"2024-04-24T23:12:28.002540+0000","last_unstale":"2024-04-24T23:12:33.992883+0000","last_undegraded":"2024-04-24T23:12:33.992883+0000","last_fullsized":"2024-04-24T23:12:33.992883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:57.792628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252793+0000","last_change":"2024-04-24T23:12:32.031160+0000","last_active":"2024-04-24T23:12:33.252793+0000","last_peered":"2024-04-24T23:12:33.252793+0000","last_clean":"2024-04-24T23:12:33.252793+0000","last_became_active":"2024-04-24T23:12:32.030719+0000","last_became_peered":"2024-04-24T23:12:32.030719+0000","last_unstale":"2024-04-24T23:12:33.252793+0000","last_undegraded":"2024-04-24T23:12:33.252793+0000","last_fullsized":"2024-04-24T23:12:33.252793+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:45.648011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:54.882483+0000","last_change":"2024-04-24T23:12:26.998802+0000","last_active":"2024-04-24T23:12:54.882483+0000","last_peered":"2024-04-24T23:12:54.882483+0000","last_clean":"2024-04-24T23:12:54.882483+0000","last_became_active":"2024-04-24T23:12:26.998471+0000","last_became_peered":"2024-04-24T23:12:26.998471+0000","last_unstale":"2024-04-24T23:12:54.882483+0000","last_undegraded":"2024-04-24T23:12:54.882483+0000","last_fullsized":"2024-04-24T23:12:54.882483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:54.576879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253965+0000","last_change":"2024-04-24T23:12:28.003211+0000","last_active":"2024-04-24T23:12:33.253965+0000","last_peered":"2024-04-24T23:12:33.253965+0000","last_clean":"2024-04-24T23:12:33.253965+0000","last_became_active":"2024-04-24T23:12:28.002897+0000","last_became_peered":"2024-04-24T23:12:28.002897+0000","last_unstale":"2024-04-24T23:12:33.253965+0000","last_undegraded":"2024-04-24T23:12:33.253965+0000","last_fullsized":"2024-04-24T23:12:33.253965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.187239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992488+0000","last_change":"2024-04-24T23:12:32.028857+0000","last_active":"2024-04-24T23:12:33.992488+0000","last_peered":"2024-04-24T23:12:33.992488+0000","last_clean":"2024-04-24T23:12:33.992488+0000","last_became_active":"2024-04-24T23:12:32.028455+0000","last_became_peered":"2024-04-24T23:12:32.028455+0000","last_unstale":"2024-04-24T23:12:33.992488+0000","last_undegraded":"2024-04-24T23:12:33.992488+0000","last_fullsized":"2024-04-24T23:12:33.992488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:23.073895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.873745+0000","last_change":"2024-04-24T23:12:27.003525+0000","last_active":"2024-04-24T23:12:53.873745+0000","last_peered":"2024-04-24T23:12:53.873745+0000","last_clean":"2024-04-24T23:12:53.873745+0000","last_became_active":"2024-04-24T23:12:27.003203+0000","last_became_peered":"2024-04-24T23:12:27.003203+0000","last_unstale":"2024-04-24T23:12:53.873745+0000","last_undegraded":"2024-04-24T23:12:53.873745+0000","last_fullsized":"2024-04-24T23:12:53.873745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:18.521925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512579+0000","last_change":"2024-04-24T23:12:28.017111+0000","last_active":"2024-04-24T23:12:33.512579+0000","last_peered":"2024-04-24T23:12:33.512579+0000","last_clean":"2024-04-24T23:12:33.512579+0000","last_became_active":"2024-04-24T23:12:28.016725+0000","last_became_peered":"2024-04-24T23:12:28.016725+0000","last_unstale":"2024-04-24T23:12:33.512579+0000","last_undegraded":"2024-04-24T23:12:33.512579+0000","last_fullsized":"2024-04-24T23:12:33.512579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:25.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992818+0000","last_change":"2024-04-24T23:12:32.027963+0000","last_active":"2024-04-24T23:12:33.992818+0000","last_peered":"2024-04-24T23:12:33.992818+0000","last_clean":"2024-04-24T23:12:33.992818+0000","last_became_active":"2024-04-24T23:12:32.027585+0000","last_became_peered":"2024-04-24T23:12:32.027585+0000","last_unstale":"2024-04-24T23:12:33.992818+0000","last_undegraded":"2024-04-24T23:12:33.992818+0000","last_fullsized":"2024-04-24T23:12:33.992818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:27.111414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.874200+0000","last_change":"2024-04-24T23:12:27.002797+0000","last_active":"2024-04-24T23:12:53.874200+0000","last_peered":"2024-04-24T23:12:53.874200+0000","last_clean":"2024-04-24T23:12:53.874200+0000","last_became_active":"2024-04-24T23:12:27.002086+0000","last_became_peered":"2024-04-24T23:12:27.002086+0000","last_unstale":"2024-04-24T23:12:53.874200+0000","last_undegraded":"2024-04-24T23:12:53.874200+0000","last_fullsized":"2024-04-24T23:12:53.874200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:55.025453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512401+0000","last_change":"2024-04-24T23:12:28.014047+0000","last_active":"2024-04-24T23:12:33.512401+0000","last_peered":"2024-04-24T23:12:33.512401+0000","last_clean":"2024-04-24T23:12:33.512401+0000","last_became_active":"2024-04-24T23:12:28.013699+0000","last_became_peered":"2024-04-24T23:12:28.013699+0000","last_unstale":"2024-04-24T23:12:33.512401+0000","last_undegraded":"2024-04-24T23:12:33.512401+0000","last_fullsized":"2024-04-24T23:12:33.512401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:51.098479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254282+0000","last_change":"2024-04-24T23:12:32.029104+0000","last_active":"2024-04-24T23:12:33.254282+0000","last_peered":"2024-04-24T23:12:33.254282+0000","last_clean":"2024-04-24T23:12:33.254282+0000","last_became_active":"2024-04-24T23:12:32.028704+0000","last_became_peered":"2024-04-24T23:12:32.028704+0000","last_unstale":"2024-04-24T23:12:33.254282+0000","last_undegraded":"2024-04-24T23:12:33.254282+0000","last_fullsized":"2024-04-24T23:12:33.254282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:33.200884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.876613+0000","last_change":"2024-04-24T23:12:27.007422+0000","last_active":"2024-04-24T23:12:53.876613+0000","last_peered":"2024-04-24T23:12:53.876613+0000","last_clean":"2024-04-24T23:12:53.876613+0000","last_became_active":"2024-04-24T23:12:27.007024+0000","last_became_peered":"2024-04-24T23:12:27.007024+0000","last_unstale":"2024-04-24T23:12:53.876613+0000","last_undegraded":"2024-04-24T23:12:53.876613+0000","last_fullsized":"2024-04-24T23:12:53.876613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:30.894232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252372+0000","last_change":"2024-04-24T23:12:28.010510+0000","last_active":"2024-04-24T23:12:33.252372+0000","last_peered":"2024-04-24T23:12:33.252372+0000","last_clean":"2024-04-24T23:12:33.252372+0000","last_became_active":"2024-04-24T23:12:28.009244+0000","last_became_peered":"2024-04-24T23:12:28.009244+0000","last_unstale":"2024-04-24T23:12:33.252372+0000","last_undegraded":"2024-04-24T23:12:33.252372+0000","last_fullsized":"2024-04-24T23:12:33.252372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.146563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252026+0000","last_change":"2024-04-24T23:12:32.031156+0000","last_active":"2024-04-24T23:12:33.252026+0000","last_peered":"2024-04-24T23:12:33.252026+0000","last_clean":"2024-04-24T23:12:33.252026+0000","last_became_active":"2024-04-24T23:12:32.030653+0000","last_became_peered":"2024-04-24T23:12:32.030653+0000","last_unstale":"2024-04-24T23:12:33.252026+0000","last_undegraded":"2024-04-24T23:12:33.252026+0000","last_fullsized":"2024-04-24T23:12:33.252026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:48.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.865655+0000","last_change":"2024-04-24T23:12:26.998003+0000","last_active":"2024-04-24T23:12:53.865655+0000","last_peered":"2024-04-24T23:12:53.865655+0000","last_clean":"2024-04-24T23:12:53.865655+0000","last_became_active":"2024-04-24T23:12:26.997661+0000","last_became_peered":"2024-04-24T23:12:26.997661+0000","last_unstale":"2024-04-24T23:12:53.865655+0000","last_undegraded":"2024-04-24T23:12:53.865655+0000","last_fullsized":"2024-04-24T23:12:53.865655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:17.995397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992273+0000","last_change":"2024-04-24T23:12:28.004822+0000","last_active":"2024-04-24T23:12:33.992273+0000","last_peered":"2024-04-24T23:12:33.992273+0000","last_clean":"2024-04-24T23:12:33.992273+0000","last_became_active":"2024-04-24T23:12:28.004334+0000","last_became_peered":"2024-04-24T23:12:28.004334+0000","last_unstale":"2024-04-24T23:12:33.992273+0000","last_undegraded":"2024-04-24T23:12:33.992273+0000","last_fullsized":"2024-04-24T23:12:33.992273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.717796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992005+0000","last_change":"2024-04-24T23:12:32.085309+0000","last_active":"2024-04-24T23:12:33.992005+0000","last_peered":"2024-04-24T23:12:33.992005+0000","last_clean":"2024-04-24T23:12:33.992005+0000","last_became_active":"2024-04-24T23:12:32.084892+0000","last_became_peered":"2024-04-24T23:12:32.084892+0000","last_unstale":"2024-04-24T23:12:33.992005+0000","last_undegraded":"2024-04-24T23:12:33.992005+0000","last_fullsized":"2024-04-24T23:12:33.992005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:20.318901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992587+0000","last_change":"2024-04-24T23:12:27.005096+0000","last_active":"2024-04-24T23:12:33.992587+0000","last_peered":"2024-04-24T23:12:33.992587+0000","last_clean":"2024-04-24T23:12:33.992587+0000","last_became_active":"2024-04-24T23:12:27.004699+0000","last_became_peered":"2024-04-24T23:12:27.004699+0000","last_unstale":"2024-04-24T23:12:33.992587+0000","last_undegraded":"2024-04-24T23:12:33.992587+0000","last_fullsized":"2024-04-24T23:12:33.992587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:58.225361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993149+0000","last_change":"2024-04-24T23:12:28.012230+0000","last_active":"2024-04-24T23:12:33.993149+0000","last_peered":"2024-04-24T23:12:33.993149+0000","last_clean":"2024-04-24T23:12:33.993149+0000","last_became_active":"2024-04-24T23:12:28.011900+0000","last_became_peered":"2024-04-24T23:12:28.011900+0000","last_unstale":"2024-04-24T23:12:33.993149+0000","last_undegraded":"2024-04-24T23:12:33.993149+0000","last_fullsized":"2024-04-24T23:12:33.993149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.780548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254018+0000","last_change":"2024-04-24T23:12:32.026388+0000","last_active":"2024-04-24T23:12:33.254018+0000","last_peered":"2024-04-24T23:12:33.254018+0000","last_clean":"2024-04-24T23:12:33.254018+0000","last_became_active":"2024-04-24T23:12:32.025994+0000","last_became_peered":"2024-04-24T23:12:32.025994+0000","last_unstale":"2024-04-24T23:12:33.254018+0000","last_undegraded":"2024-04-24T23:12:33.254018+0000","last_fullsized":"2024-04-24T23:12:33.254018+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:13.062781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:54.881689+0000","last_change":"2024-04-24T23:12:27.005080+0000","last_active":"2024-04-24T23:12:54.881689+0000","last_peered":"2024-04-24T23:12:54.881689+0000","last_clean":"2024-04-24T23:12:54.881689+0000","last_became_active":"2024-04-24T23:12:27.004740+0000","last_became_peered":"2024-04-24T23:12:27.004740+0000","last_unstale":"2024-04-24T23:12:54.881689+0000","last_undegraded":"2024-04-24T23:12:54.881689+0000","last_fullsized":"2024-04-24T23:12:54.881689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:22.502399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992913+0000","last_change":"2024-04-24T23:12:28.008016+0000","last_active":"2024-04-24T23:12:33.992913+0000","last_peered":"2024-04-24T23:12:33.992913+0000","last_clean":"2024-04-24T23:12:33.992913+0000","last_became_active":"2024-04-24T23:12:28.007139+0000","last_became_peered":"2024-04-24T23:12:28.007139+0000","last_unstale":"2024-04-24T23:12:33.992913+0000","last_undegraded":"2024-04-24T23:12:33.992913+0000","last_fullsized":"2024-04-24T23:12:33.992913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.698193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254090+0000","last_change":"2024-04-24T23:12:32.085290+0000","last_active":"2024-04-24T23:12:33.254090+0000","last_peered":"2024-04-24T23:12:33.254090+0000","last_clean":"2024-04-24T23:12:33.254090+0000","last_became_active":"2024-04-24T23:12:32.084852+0000","last_became_peered":"2024-04-24T23:12:32.084852+0000","last_unstale":"2024-04-24T23:12:33.254090+0000","last_undegraded":"2024-04-24T23:12:33.254090+0000","last_fullsized":"2024-04-24T23:12:33.254090+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.753633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252226+0000","last_change":"2024-04-24T23:12:28.007552+0000","last_active":"2024-04-24T23:12:33.252226+0000","last_peered":"2024-04-24T23:12:33.252226+0000","last_clean":"2024-04-24T23:12:33.252226+0000","last_became_active":"2024-04-24T23:12:28.007196+0000","last_became_peered":"2024-04-24T23:12:28.007196+0000","last_unstale":"2024-04-24T23:12:33.252226+0000","last_undegraded":"2024-04-24T23:12:33.252226+0000","last_fullsized":"2024-04-24T23:12:33.252226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:15.695485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:52.900449+0000","last_change":"2024-04-24T23:12:26.990814+0000","last_active":"2024-04-24T23:12:52.900449+0000","last_peered":"2024-04-24T23:12:52.900449+0000","last_clean":"2024-04-24T23:12:52.900449+0000","last_became_active":"2024-04-24T23:12:26.990476+0000","last_became_peered":"2024-04-24T23:12:26.990476+0000","last_unstale":"2024-04-24T23:12:52.900449+0000","last_undegraded":"2024-04-24T23:12:52.900449+0000","last_fullsized":"2024-04-24T23:12:52.900449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:32.645770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:35.626068+0000","last_change":"2024-04-24T23:12:32.022039+0000","last_active":"2024-04-24T23:12:35.626068+0000","last_peered":"2024-04-24T23:12:35.626068+0000","last_clean":"2024-04-24T23:12:35.626068+0000","last_became_active":"2024-04-24T23:12:32.021514+0000","last_became_peered":"2024-04-24T23:12:32.021514+0000","last_unstale":"2024-04-24T23:12:35.626068+0000","last_undegraded":"2024-04-24T23:12:35.626068+0000","last_fullsized":"2024-04-24T23:12:35.626068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:42.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993211+0000","last_change":"2024-04-24T23:12:28.008676+0000","last_active":"2024-04-24T23:12:33.993211+0000","last_peered":"2024-04-24T23:12:33.993211+0000","last_clean":"2024-04-24T23:12:33.993211+0000","last_became_active":"2024-04-24T23:12:28.008376+0000","last_became_peered":"2024-04-24T23:12:28.008376+0000","last_unstale":"2024-04-24T23:12:33.993211+0000","last_undegraded":"2024-04-24T23:12:33.993211+0000","last_fullsized":"2024-04-24T23:12:33.993211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:15.358332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.874131+0000","last_change":"2024-04-24T23:12:26.994952+0000","last_active":"2024-04-24T23:12:53.874131+0000","last_peered":"2024-04-24T23:12:53.874131+0000","last_clean":"2024-04-24T23:12:53.874131+0000","last_became_active":"2024-04-24T23:12:26.994556+0000","last_became_peered":"2024-04-24T23:12:26.994556+0000","last_unstale":"2024-04-24T23:12:53.874131+0000","last_undegraded":"2024-04-24T23:12:53.874131+0000","last_fullsized":"2024-04-24T23:12:53.874131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.257427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512254+0000","last_change":"2024-04-24T23:11:16.721876+0000","last_active":"2024-04-24T23:12:33.512254+0000","last_peered":"2024-04-24T23:12:33.512254+0000","last_clean":"2024-04-24T23:12:33.512254+0000","last_became_active":"2024-04-24T23:11:16.721291+0000","last_became_peered":"2024-04-24T23:11:16.721291+0000","last_unstale":"2024-04-24T23:12:33.512254+0000","last_undegraded":"2024-04-24T23:12:33.512254+0000","last_fullsized":"2024-04-24T23:12:33.512254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_clean_scrub_stamp":"2024-04-24T23:07:51.257753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.458939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252459+0000","last_change":"2024-04-24T23:12:32.085905+0000","last_active":"2024-04-24T23:12:33.252459+0000","last_peered":"2024-04-24T23:12:33.252459+0000","last_clean":"2024-04-24T23:12:33.252459+0000","last_became_active":"2024-04-24T23:12:32.085374+0000","last_became_peered":"2024-04-24T23:12:32.085374+0000","last_unstale":"2024-04-24T23:12:33.252459+0000","last_undegraded":"2024-04-24T23:12:33.252459+0000","last_fullsized":"2024-04-24T23:12:33.252459+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:16.290651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253498+0000","last_change":"2024-04-24T23:12:27.999880+0000","last_active":"2024-04-24T23:12:33.253498+0000","last_peered":"2024-04-24T23:12:33.253498+0000","last_clean":"2024-04-24T23:12:33.253498+0000","last_became_active":"2024-04-24T23:12:27.999543+0000","last_became_peered":"2024-04-24T23:12:27.999543+0000","last_unstale":"2024-04-24T23:12:33.253498+0000","last_undegraded":"2024-04-24T23:12:33.253498+0000","last_fullsized":"2024-04-24T23:12:33.253498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:44.758968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:52.899686+0000","last_change":"2024-04-24T23:12:27.005918+0000","last_active":"2024-04-24T23:12:52.899686+0000","last_peered":"2024-04-24T23:12:52.899686+0000","last_clean":"2024-04-24T23:12:52.899686+0000","last_became_active":"2024-04-24T23:12:27.005522+0000","last_became_peered":"2024-04-24T23:12:27.005522+0000","last_unstale":"2024-04-24T23:12:52.899686+0000","last_undegraded":"2024-04-24T23:12:52.899686+0000","last_fullsized":"2024-04-24T23:12:52.899686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:14.292662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992251+0000","last_change":"2024-04-24T23:12:32.024952+0000","last_active":"2024-04-24T23:12:33.992251+0000","last_peered":"2024-04-24T23:12:33.992251+0000","last_clean":"2024-04-24T23:12:33.992251+0000","last_became_active":"2024-04-24T23:12:32.024404+0000","last_became_peered":"2024-04-24T23:12:32.024404+0000","last_unstale":"2024-04-24T23:12:33.992251+0000","last_undegraded":"2024-04-24T23:12:33.992251+0000","last_fullsized":"2024-04-24T23:12:33.992251+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.377794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.876476+0000","last_change":"2024-04-24T23:12:27.007191+0000","last_active":"2024-04-24T23:12:53.876476+0000","last_peered":"2024-04-24T23:12:53.876476+0000","last_clean":"2024-04-24T23:12:53.876476+0000","last_became_active":"2024-04-24T23:12:27.006867+0000","last_became_peered":"2024-04-24T23:12:27.006867+0000","last_unstale":"2024-04-24T23:12:53.876476+0000","last_undegraded":"2024-04-24T23:12:53.876476+0000","last_fullsized":"2024-04-24T23:12:53.876476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:58.112293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254082+0000","last_change":"2024-04-24T23:12:27.996322+0000","last_active":"2024-04-24T23:12:33.254082+0000","last_peered":"2024-04-24T23:12:33.254082+0000","last_clean":"2024-04-24T23:12:33.254082+0000","last_became_active":"2024-04-24T23:12:27.995991+0000","last_became_peered":"2024-04-24T23:12:27.995991+0000","last_unstale":"2024-04-24T23:12:33.254082+0000","last_undegraded":"2024-04-24T23:12:33.254082+0000","last_fullsized":"2024-04-24T23:12:33.254082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:08.960347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991531+0000","last_change":"2024-04-24T23:12:32.084959+0000","last_active":"2024-04-24T23:12:33.991531+0000","last_peered":"2024-04-24T23:12:33.991531+0000","last_clean":"2024-04-24T23:12:33.991531+0000","last_became_active":"2024-04-24T23:12:32.084553+0000","last_became_peered":"2024-04-24T23:12:32.084553+0000","last_unstale":"2024-04-24T23:12:33.991531+0000","last_undegraded":"2024-04-24T23:12:33.991531+0000","last_fullsized":"2024-04-24T23:12:33.991531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.836337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993365+0000","last_change":"2024-04-24T23:12:27.007742+0000","last_active":"2024-04-24T23:12:33.993365+0000","last_peered":"2024-04-24T23:12:33.993365+0000","last_clean":"2024-04-24T23:12:33.993365+0000","last_became_active":"2024-04-24T23:12:27.007423+0000","last_became_peered":"2024-04-24T23:12:27.007423+0000","last_unstale":"2024-04-24T23:12:33.993365+0000","last_undegraded":"2024-04-24T23:12:33.993365+0000","last_fullsized":"2024-04-24T23:12:33.993365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:39.478356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992351+0000","last_change":"2024-04-24T23:12:28.010961+0000","last_active":"2024-04-24T23:12:33.992351+0000","last_peered":"2024-04-24T23:12:33.992351+0000","last_clean":"2024-04-24T23:12:33.992351+0000","last_became_active":"2024-04-24T23:12:28.010641+0000","last_became_peered":"2024-04-24T23:12:28.010641+0000","last_unstale":"2024-04-24T23:12:33.992351+0000","last_undegraded":"2024-04-24T23:12:33.992351+0000","last_fullsized":"2024-04-24T23:12:33.992351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:43.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992679+0000","last_change":"2024-04-24T23:12:32.023687+0000","last_active":"2024-04-24T23:12:33.992679+0000","last_peered":"2024-04-24T23:12:33.992679+0000","last_clean":"2024-04-24T23:12:33.992679+0000","last_became_active":"2024-04-24T23:12:32.023052+0000","last_became_peered":"2024-04-24T23:12:32.023052+0000","last_unstale":"2024-04-24T23:12:33.992679+0000","last_undegraded":"2024-04-24T23:12:33.992679+0000","last_fullsized":"2024-04-24T23:12:33.992679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:39.854030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:54.880711+0000","last_change":"2024-04-24T23:12:27.009743+0000","last_active":"2024-04-24T23:12:54.880711+0000","last_peered":"2024-04-24T23:12:54.880711+0000","last_clean":"2024-04-24T23:12:54.880711+0000","last_became_active":"2024-04-24T23:12:27.009343+0000","last_became_peered":"2024-04-24T23:12:27.009343+0000","last_unstale":"2024-04-24T23:12:54.880711+0000","last_undegraded":"2024-04-24T23:12:54.880711+0000","last_fullsized":"2024-04-24T23:12:54.880711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:20.387738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991705+0000","last_change":"2024-04-24T23:12:28.011895+0000","last_active":"2024-04-24T23:12:33.991705+0000","last_peered":"2024-04-24T23:12:33.991705+0000","last_clean":"2024-04-24T23:12:33.991705+0000","last_became_active":"2024-04-24T23:12:28.011546+0000","last_became_peered":"2024-04-24T23:12:28.011546+0000","last_unstale":"2024-04-24T23:12:33.991705+0000","last_undegraded":"2024-04-24T23:12:33.991705+0000","last_fullsized":"2024-04-24T23:12:33.991705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:17.982200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992180+0000","last_change":"2024-04-24T23:12:32.086209+0000","last_active":"2024-04-24T23:12:33.992180+0000","last_peered":"2024-04-24T23:12:33.992180+0000","last_clean":"2024-04-24T23:12:33.992180+0000","last_became_active":"2024-04-24T23:12:32.085752+0000","last_became_peered":"2024-04-24T23:12:32.085752+0000","last_unstale":"2024-04-24T23:12:33.992180+0000","last_undegraded":"2024-04-24T23:12:33.992180+0000","last_fullsized":"2024-04-24T23:12:33.992180+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:52.120905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:51.886798+0000","last_change":"2024-04-24T23:12:26.996355+0000","last_active":"2024-04-24T23:12:51.886798+0000","last_peered":"2024-04-24T23:12:51.886798+0000","last_clean":"2024-04-24T23:12:51.886798+0000","last_became_active":"2024-04-24T23:12:26.995486+0000","last_became_peered":"2024-04-24T23:12:26.995486+0000","last_unstale":"2024-04-24T23:12:51.886798+0000","last_undegraded":"2024-04-24T23:12:51.886798+0000","last_fullsized":"2024-04-24T23:12:51.886798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:48.981985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991778+0000","last_change":"2024-04-24T23:12:28.004610+0000","last_active":"2024-04-24T23:12:33.991778+0000","last_peered":"2024-04-24T23:12:33.991778+0000","last_clean":"2024-04-24T23:12:33.991778+0000","last_became_active":"2024-04-24T23:12:28.004051+0000","last_became_peered":"2024-04-24T23:12:28.004051+0000","last_unstale":"2024-04-24T23:12:33.991778+0000","last_undegraded":"2024-04-24T23:12:33.991778+0000","last_fullsized":"2024-04-24T23:12:33.991778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:42.656954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992985+0000","last_change":"2024-04-24T23:12:32.029269+0000","last_active":"2024-04-24T23:12:33.992985+0000","last_peered":"2024-04-24T23:12:33.992985+0000","last_clean":"2024-04-24T23:12:33.992985+0000","last_became_active":"2024-04-24T23:12:32.028880+0000","last_became_peered":"2024-04-24T23:12:32.028880+0000","last_unstale":"2024-04-24T23:12:33.992985+0000","last_undegraded":"2024-04-24T23:12:33.992985+0000","last_fullsized":"2024-04-24T23:12:33.992985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:19.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.873415+0000","last_change":"2024-04-24T23:12:27.006357+0000","last_active":"2024-04-24T23:12:53.873415+0000","last_peered":"2024-04-24T23:12:53.873415+0000","last_clean":"2024-04-24T23:12:53.873415+0000","last_became_active":"2024-04-24T23:12:27.006040+0000","last_became_peered":"2024-04-24T23:12:27.006040+0000","last_unstale":"2024-04-24T23:12:53.873415+0000","last_undegraded":"2024-04-24T23:12:53.873415+0000","last_fullsized":"2024-04-24T23:12:53.873415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:27.520663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992003+0000","last_change":"2024-04-24T23:12:28.008909+0000","last_active":"2024-04-24T23:12:33.992003+0000","last_peered":"2024-04-24T23:12:33.992003+0000","last_clean":"2024-04-24T23:12:33.992003+0000","last_became_active":"2024-04-24T23:12:28.008552+0000","last_became_peered":"2024-04-24T23:12:28.008552+0000","last_unstale":"2024-04-24T23:12:33.992003+0000","last_undegraded":"2024-04-24T23:12:33.992003+0000","last_fullsized":"2024-04-24T23:12:33.992003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:20.628234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992614+0000","last_change":"2024-04-24T23:12:32.029720+0000","last_active":"2024-04-24T23:12:33.992614+0000","last_peered":"2024-04-24T23:12:33.992614+0000","last_clean":"2024-04-24T23:12:33.992614+0000","last_became_active":"2024-04-24T23:12:32.029311+0000","last_became_peered":"2024-04-24T23:12:32.029311+0000","last_unstale":"2024-04-24T23:12:33.992614+0000","last_undegraded":"2024-04-24T23:12:33.992614+0000","last_fullsized":"2024-04-24T23:12:33.992614+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:37.832102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.883005+0000","last_change":"2024-04-24T23:12:26.995552+0000","last_active":"2024-04-24T23:12:53.883005+0000","last_peered":"2024-04-24T23:12:53.883005+0000","last_clean":"2024-04-24T23:12:53.883005+0000","last_became_active":"2024-04-24T23:12:26.994922+0000","last_became_peered":"2024-04-24T23:12:26.994922+0000","last_unstale":"2024-04-24T23:12:53.883005+0000","last_undegraded":"2024-04-24T23:12:53.883005+0000","last_fullsized":"2024-04-24T23:12:53.883005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:17.981344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992726+0000","last_change":"2024-04-24T23:12:28.008802+0000","last_active":"2024-04-24T23:12:33.992726+0000","last_peered":"2024-04-24T23:12:33.992726+0000","last_clean":"2024-04-24T23:12:33.992726+0000","last_became_active":"2024-04-24T23:12:28.008496+0000","last_became_peered":"2024-04-24T23:12:28.008496+0000","last_unstale":"2024-04-24T23:12:33.992726+0000","last_undegraded":"2024-04-24T23:12:33.992726+0000","last_fullsized":"2024-04-24T23:12:33.992726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:54.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992280+0000","last_change":"2024-04-24T23:12:32.085555+0000","last_active":"2024-04-24T23:12:33.992280+0000","last_peered":"2024-04-24T23:12:33.992280+0000","last_clean":"2024-04-24T23:12:33.992280+0000","last_became_active":"2024-04-24T23:12:32.085085+0000","last_became_peered":"2024-04-24T23:12:32.085085+0000","last_unstale":"2024-04-24T23:12:33.992280+0000","last_undegraded":"2024-04-24T23:12:33.992280+0000","last_fullsized":"2024-04-24T23:12:33.992280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:24.563964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:51.888841+0000","last_change":"2024-04-24T23:12:27.003221+0000","last_active":"2024-04-24T23:12:51.888841+0000","last_peered":"2024-04-24T23:12:51.888841+0000","last_clean":"2024-04-24T23:12:51.888841+0000","last_became_active":"2024-04-24T23:12:27.002879+0000","last_became_peered":"2024-04-24T23:12:27.002879+0000","last_unstale":"2024-04-24T23:12:51.888841+0000","last_undegraded":"2024-04-24T23:12:51.888841+0000","last_fullsized":"2024-04-24T23:12:51.888841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:10:10.548202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512526+0000","last_change":"2024-04-24T23:12:28.014865+0000","last_active":"2024-04-24T23:12:33.512526+0000","last_peered":"2024-04-24T23:12:33.512526+0000","last_clean":"2024-04-24T23:12:33.512526+0000","last_became_active":"2024-04-24T23:12:28.014538+0000","last_became_peered":"2024-04-24T23:12:28.014538+0000","last_unstale":"2024-04-24T23:12:33.512526+0000","last_undegraded":"2024-04-24T23:12:33.512526+0000","last_fullsized":"2024-04-24T23:12:33.512526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:55.306883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256077+0000","last_change":"2024-04-24T23:12:32.024849+0000","last_active":"2024-04-24T23:12:33.256077+0000","last_peered":"2024-04-24T23:12:33.256077+0000","last_clean":"2024-04-24T23:12:33.256077+0000","last_became_active":"2024-04-24T23:12:32.024412+0000","last_became_peered":"2024-04-24T23:12:32.024412+0000","last_unstale":"2024-04-24T23:12:33.256077+0000","last_undegraded":"2024-04-24T23:12:33.256077+0000","last_fullsized":"2024-04-24T23:12:33.256077+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:16.884565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.876188+0000","last_change":"2024-04-24T23:12:27.007284+0000","last_active":"2024-04-24T23:12:53.876188+0000","last_peered":"2024-04-24T23:12:53.876188+0000","last_clean":"2024-04-24T23:12:53.876188+0000","last_became_active":"2024-04-24T23:12:27.006916+0000","last_became_peered":"2024-04-24T23:12:27.006916+0000","last_unstale":"2024-04-24T23:12:53.876188+0000","last_undegraded":"2024-04-24T23:12:53.876188+0000","last_fullsized":"2024-04-24T23:12:53.876188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:40.333387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512970+0000","last_change":"2024-04-24T23:12:28.011603+0000","last_active":"2024-04-24T23:12:33.512970+0000","last_peered":"2024-04-24T23:12:33.512970+0000","last_clean":"2024-04-24T23:12:33.512970+0000","last_became_active":"2024-04-24T23:12:28.011268+0000","last_became_peered":"2024-04-24T23:12:28.011268+0000","last_unstale":"2024-04-24T23:12:33.512970+0000","last_undegraded":"2024-04-24T23:12:33.512970+0000","last_fullsized":"2024-04-24T23:12:33.512970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.693024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991494+0000","last_change":"2024-04-24T23:12:32.033965+0000","last_active":"2024-04-24T23:12:33.991494+0000","last_peered":"2024-04-24T23:12:33.991494+0000","last_clean":"2024-04-24T23:12:33.991494+0000","last_became_active":"2024-04-24T23:12:32.033472+0000","last_became_peered":"2024-04-24T23:12:32.033472+0000","last_unstale":"2024-04-24T23:12:33.991494+0000","last_undegraded":"2024-04-24T23:12:33.991494+0000","last_fullsized":"2024-04-24T23:12:33.991494+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:05.243047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:53.874251+0000","last_change":"2024-04-24T23:12:26.991981+0000","last_active":"2024-04-24T23:12:53.874251+0000","last_peered":"2024-04-24T23:12:53.874251+0000","last_clean":"2024-04-24T23:12:53.874251+0000","last_became_active":"2024-04-24T23:12:26.990707+0000","last_became_peered":"2024-04-24T23:12:26.990707+0000","last_unstale":"2024-04-24T23:12:53.874251+0000","last_undegraded":"2024-04-24T23:12:53.874251+0000","last_fullsized":"2024-04-24T23:12:53.874251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:36.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512770+0000","last_change":"2024-04-24T23:12:28.010428+0000","last_active":"2024-04-24T23:12:33.512770+0000","last_peered":"2024-04-24T23:12:33.512770+0000","last_clean":"2024-04-24T23:12:33.512770+0000","last_became_active":"2024-04-24T23:12:28.010096+0000","last_became_peered":"2024-04-24T23:12:28.010096+0000","last_unstale":"2024-04-24T23:12:33.512770+0000","last_undegraded":"2024-04-24T23:12:33.512770+0000","last_fullsized":"2024-04-24T23:12:33.512770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:31.949617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.251812+0000","last_change":"2024-04-24T23:12:32.023828+0000","last_active":"2024-04-24T23:12:33.251812+0000","last_peered":"2024-04-24T23:12:33.251812+0000","last_clean":"2024-04-24T23:12:33.251812+0000","last_became_active":"2024-04-24T23:12:32.023437+0000","last_became_peered":"2024-04-24T23:12:32.023437+0000","last_unstale":"2024-04-24T23:12:33.251812+0000","last_undegraded":"2024-04-24T23:12:33.251812+0000","last_fullsized":"2024-04-24T23:12:33.251812+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.186860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254199+0000","last_change":"2024-04-24T23:12:26.993711+0000","last_active":"2024-04-24T23:12:33.254199+0000","last_peered":"2024-04-24T23:12:33.254199+0000","last_clean":"2024-04-24T23:12:33.254199+0000","last_became_active":"2024-04-24T23:12:26.993368+0000","last_became_peered":"2024-04-24T23:12:26.993368+0000","last_unstale":"2024-04-24T23:12:33.254199+0000","last_undegraded":"2024-04-24T23:12:33.254199+0000","last_fullsized":"2024-04-24T23:12:33.254199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:58.843747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.512964+0000","last_change":"2024-04-24T23:12:28.003026+0000","last_active":"2024-04-24T23:12:33.512964+0000","last_peered":"2024-04-24T23:12:33.512964+0000","last_clean":"2024-04-24T23:12:33.512964+0000","last_became_active":"2024-04-24T23:12:28.002693+0000","last_became_peered":"2024-04-24T23:12:28.002693+0000","last_unstale":"2024-04-24T23:12:33.512964+0000","last_undegraded":"2024-04-24T23:12:33.512964+0000","last_fullsized":"2024-04-24T23:12:33.512964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:30.926251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992017+0000","last_change":"2024-04-24T23:12:28.013414+0000","last_active":"2024-04-24T23:12:33.992017+0000","last_peered":"2024-04-24T23:12:33.992017+0000","last_clean":"2024-04-24T23:12:33.992017+0000","last_became_active":"2024-04-24T23:12:28.013030+0000","last_became_peered":"2024-04-24T23:12:28.013030+0000","last_unstale":"2024-04-24T23:12:33.992017+0000","last_undegraded":"2024-04-24T23:12:33.992017+0000","last_fullsized":"2024-04-24T23:12:33.992017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:48.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993146+0000","last_change":"2024-04-24T23:12:28.004821+0000","last_active":"2024-04-24T23:12:33.993146+0000","last_peered":"2024-04-24T23:12:33.993146+0000","last_clean":"2024-04-24T23:12:33.993146+0000","last_became_active":"2024-04-24T23:12:28.003162+0000","last_became_peered":"2024-04-24T23:12:28.003162+0000","last_unstale":"2024-04-24T23:12:33.993146+0000","last_undegraded":"2024-04-24T23:12:33.993146+0000","last_fullsized":"2024-04-24T23:12:33.993146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:59.109665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992356+0000","last_change":"2024-04-24T23:12:28.002115+0000","last_active":"2024-04-24T23:12:33.992356+0000","last_peered":"2024-04-24T23:12:33.992356+0000","last_clean":"2024-04-24T23:12:33.992356+0000","last_became_active":"2024-04-24T23:12:28.001727+0000","last_became_peered":"2024-04-24T23:12:28.001727+0000","last_unstale":"2024-04-24T23:12:33.992356+0000","last_undegraded":"2024-04-24T23:12:33.992356+0000","last_fullsized":"2024-04-24T23:12:33.992356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:14.348843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.991577+0000","last_change":"2024-04-24T23:12:28.007784+0000","last_active":"2024-04-24T23:12:33.991577+0000","last_peered":"2024-04-24T23:12:33.991577+0000","last_clean":"2024-04-24T23:12:33.991577+0000","last_became_active":"2024-04-24T23:12:28.007481+0000","last_became_peered":"2024-04-24T23:12:28.007481+0000","last_unstale":"2024-04-24T23:12:33.991577+0000","last_undegraded":"2024-04-24T23:12:33.991577+0000","last_fullsized":"2024-04-24T23:12:33.991577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:33.753448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.254000+0000","last_change":"2024-04-24T23:12:28.009100+0000","last_active":"2024-04-24T23:12:33.254000+0000","last_peered":"2024-04-24T23:12:33.254000+0000","last_clean":"2024-04-24T23:12:33.254000+0000","last_became_active":"2024-04-24T23:12:28.008782+0000","last_became_peered":"2024-04-24T23:12:28.008782+0000","last_unstale":"2024-04-24T23:12:33.254000+0000","last_undegraded":"2024-04-24T23:12:33.254000+0000","last_fullsized":"2024-04-24T23:12:33.254000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:44.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992502+0000","last_change":"2024-04-24T23:12:28.008129+0000","last_active":"2024-04-24T23:12:33.992502+0000","last_peered":"2024-04-24T23:12:33.992502+0000","last_clean":"2024-04-24T23:12:33.992502+0000","last_became_active":"2024-04-24T23:12:28.007819+0000","last_became_peered":"2024-04-24T23:12:28.007819+0000","last_unstale":"2024-04-24T23:12:33.992502+0000","last_undegraded":"2024-04-24T23:12:33.992502+0000","last_fullsized":"2024-04-24T23:12:33.992502+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.234705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.993247+0000","last_change":"2024-04-24T23:12:27.993188+0000","last_active":"2024-04-24T23:12:33.993247+0000","last_peered":"2024-04-24T23:12:33.993247+0000","last_clean":"2024-04-24T23:12:33.993247+0000","last_became_active":"2024-04-24T23:12:27.992832+0000","last_became_peered":"2024-04-24T23:12:27.992832+0000","last_unstale":"2024-04-24T23:12:33.993247+0000","last_undegraded":"2024-04-24T23:12:33.993247+0000","last_fullsized":"2024-04-24T23:12:33.993247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:58.623164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992528+0000","last_change":"2024-04-24T23:12:28.004099+0000","last_active":"2024-04-24T23:12:33.992528+0000","last_peered":"2024-04-24T23:12:33.992528+0000","last_clean":"2024-04-24T23:12:33.992528+0000","last_became_active":"2024-04-24T23:12:28.003802+0000","last_became_peered":"2024-04-24T23:12:28.003802+0000","last_unstale":"2024-04-24T23:12:33.992528+0000","last_undegraded":"2024-04-24T23:12:33.992528+0000","last_fullsized":"2024-04-24T23:12:33.992528+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:25.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.252100+0000","last_change":"2024-04-24T23:12:28.007700+0000","last_active":"2024-04-24T23:12:33.252100+0000","last_peered":"2024-04-24T23:12:33.252100+0000","last_clean":"2024-04-24T23:12:33.252100+0000","last_became_active":"2024-04-24T23:12:28.002923+0000","last_became_peered":"2024-04-24T23:12:28.002923+0000","last_unstale":"2024-04-24T23:12:33.252100+0000","last_undegraded":"2024-04-24T23:12:33.252100+0000","last_fullsized":"2024-04-24T23:12:33.252100+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:46.736096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992156+0000","last_change":"2024-04-24T23:12:28.002194+0000","last_active":"2024-04-24T23:12:33.992156+0000","last_peered":"2024-04-24T23:12:33.992156+0000","last_clean":"2024-04-24T23:12:33.992156+0000","last_became_active":"2024-04-24T23:12:28.001703+0000","last_became_peered":"2024-04-24T23:12:28.001703+0000","last_unstale":"2024-04-24T23:12:33.992156+0000","last_undegraded":"2024-04-24T23:12:33.992156+0000","last_fullsized":"2024-04-24T23:12:33.992156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:20.887025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.256368+0000","last_change":"2024-04-24T23:12:28.015980+0000","last_active":"2024-04-24T23:12:33.256368+0000","last_peered":"2024-04-24T23:12:33.256368+0000","last_clean":"2024-04-24T23:12:33.256368+0000","last_became_active":"2024-04-24T23:12:28.013510+0000","last_became_peered":"2024-04-24T23:12:28.013510+0000","last_unstale":"2024-04-24T23:12:33.256368+0000","last_undegraded":"2024-04-24T23:12:33.256368+0000","last_fullsized":"2024-04-24T23:12:33.256368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:37.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.253879+0000","last_change":"2024-04-24T23:12:28.001883+0000","last_active":"2024-04-24T23:12:33.253879+0000","last_peered":"2024-04-24T23:12:33.253879+0000","last_clean":"2024-04-24T23:12:33.253879+0000","last_became_active":"2024-04-24T23:12:28.001499+0000","last_became_peered":"2024-04-24T23:12:28.001499+0000","last_unstale":"2024-04-24T23:12:33.253879+0000","last_undegraded":"2024-04-24T23:12:33.253879+0000","last_fullsized":"2024-04-24T23:12:33.253879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:46.805548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-24T23:12:33.992529+0000","last_change":"2024-04-24T23:12:28.004660+0000","last_active":"2024-04-24T23:12:33.992529+0000","last_peered":"2024-04-24T23:12:33.992529+0000","last_clean":"2024-04-24T23:12:33.992529+0000","last_became_active":"2024-04-24T23:12:28.004328+0000","last_became_peered":"2024-04-24T23:12:28.004328+0000","last_unstale":"2024-04-24T23:12:33.992529+0000","last_undegraded":"2024-04-24T23:12:33.992529+0000","last_fullsized":"2024-04-24T23:12:33.992529+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:04.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10453,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":23,"num_read_kb":19,"num_write":171,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":352256,"data_stored":64404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":482291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1634304,"data_stored":1077497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776156,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":482493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037793,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":487305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1079028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462955,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":485551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":483187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":507213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":482117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":482375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724593,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1638400,"data_stored":1079384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:12:56.905 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-24T23:12:56.906 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T23:12:56.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T23:12:57.180 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:12:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:56 smithi187 ceph-mon[33292]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:56 smithi187 ceph-mon[33292]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:56 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] up:active 2024-04-24T23:12:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:56 smithi187 ceph-mon[33292]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-24T23:12:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:56 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:12:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:56 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/763080509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:57.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:56 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3840977061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:56 smithi112 ceph-mon[29186]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:56 smithi112 ceph-mon[29186]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:56 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] up:active 2024-04-24T23:12:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:56 smithi112 ceph-mon[29186]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-24T23:12:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:56 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:12:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:56 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/763080509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:56 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3840977061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:57.318 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: 2024-04-24T23:12:56.848+0000 7f8e11d89700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-24T23:12:57.318 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: 2024-04-24T23:12:56.848+0000 7f8e11d89700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-24T23:12:57.318 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: 2024-04-24T23:12:56.848+0000 7f8e11d89700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-24T23:12:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-mon[32247]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-mon[32247]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] up:active 2024-04-24T23:12:57.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-mon[32247]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-24T23:12:57.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:12:57.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/763080509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:12:57.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:56 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3840977061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:12:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:57 smithi187 ceph-mon[33292]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:57 smithi187 ceph-mon[33292]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-24T23:12:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:57 smithi187 ceph-mon[33292]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:57 smithi112 ceph-mon[29186]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:57 smithi112 ceph-mon[29186]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-24T23:12:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:57 smithi112 ceph-mon[29186]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:57 smithi150 ceph-mon[32247]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:57 smithi150 ceph-mon[32247]: pgmap v305: 129 pgs: 129 active+clean; 587 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-24T23:12:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:57 smithi150 ceph-mon[32247]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:12:58.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.869+0000 7f17e8d2e700 1 -- 172.21.15.112:0/3158187665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41023b0 msgr2=0x7f17e4102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:58.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.869+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/3158187665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41023b0 0x7f17e4102810 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f17d40099c0 tx=0x7f17d4009cd0 comp rx=0 tx=0).stop 2024-04-24T23:12:58.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 -- 172.21.15.112:0/3158187665 shutdown_connections 2024-04-24T23:12:58.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/3158187665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17e4102d50 0x7f17e4109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:58.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/3158187665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17e41011b0 0x7f17e41015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:58.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/3158187665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41023b0 0x7f17e4102810 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:58.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 -- 172.21.15.112:0/3158187665 >> 172.21.15.112:0/3158187665 conn(0x7f17e40fc750 msgr2=0x7f17e40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 -- 172.21.15.112:0/3158187665 shutdown_connections 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 -- 172.21.15.112:0/3158187665 wait complete. 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 Processor -- start 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.870+0000 7f17e8d2e700 1 -- start start 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e8d2e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41011b0 0x7f17e41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e8d2e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17e41023b0 0x7f17e41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e8d2e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17e4102d50 0x7f17e419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e8d2e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17e410cf20 con 0x7f17e41011b0 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e8d2e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f17e410cda0 con 0x7f17e41023b0 2024-04-24T23:12:58.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e8d2e700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17e410d0a0 con 0x7f17e4102d50 2024-04-24T23:12:59.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41011b0 0x7f17e41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:59.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41011b0 0x7f17e41973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38022/0 (socket says 172.21.15.112:38022) 2024-04-24T23:12:59.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 -- 172.21.15.112:0/675675517 learned_addr learned my addr 172.21.15.112:0/675675517 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:12:59.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e1d9b700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17e41023b0 0x7f17e41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e2d9d700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17e4102d50 0x7f17e419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 -- 172.21.15.112:0/675675517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17e4102d50 msgr2=0x7f17e419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17e4102d50 0x7f17e419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 -- 172.21.15.112:0/675675517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17e41023b0 msgr2=0x7f17e41978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17e41023b0 0x7f17e41978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 -- 172.21.15.112:0/675675517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e419f000 con 0x7f17e41011b0 2024-04-24T23:12:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e259c700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41011b0 0x7f17e41973a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f17d800d9b0 tx=0x7f17d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:59.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17d37fe700 1 -- 172.21.15.112:0/675675517 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8016070 con 0x7f17e41011b0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e419f2f0 con 0x7f17e41011b0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17d37fe700 1 -- 172.21.15.112:0/675675517 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f17d800be50 con 0x7f17e41011b0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17d37fe700 1 -- 172.21.15.112:0/675675517 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d800f3c0 con 0x7f17e41011b0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.871+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e419f830 con 0x7f17e41011b0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.872+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e40507a0 con 0x7f17e41011b0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.873+0000 7f17d37fe700 1 -- 172.21.15.112:0/675675517 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f17d8019070 con 0x7f17e41011b0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.874+0000 7f17d37fe700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f17cc0a2d60 0x7f17cc0a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.874+0000 7f17e1d9b700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f17cc0a2d60 0x7f17cc0a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.874+0000 7f17d37fe700 1 -- 172.21.15.112:0/675675517 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f17d80cefe0 con 0x7f17e41011b0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.874+0000 7f17e1d9b700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f17cc0a2d60 0x7f17cc0a5220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006010 tx=0x7f17d4005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:59.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:58.879+0000 7f17d37fe700 1 -- 172.21.15.112:0/675675517 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f17d8020030 con 0x7f17e41011b0 2024-04-24T23:12:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.165+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f17e402dbd0 con 0x7f17e41011b0 2024-04-24T23:12:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.168+0000 7f17d37fe700 1 -- 172.21.15.112:0/675675517 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f17e402dbd0 con 0x7f17e41011b0 2024-04-24T23:12:59.168 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-04-24T23:12:59.169 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQD7kSlmWBfqCRAAM4vCv7hkcx5RktOEz5pgww== 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f17cc0a2d60 msgr2=0x7f17cc0a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f17cc0a2d60 0x7f17cc0a5220 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17d4006010 tx=0x7f17d4005bc0 comp rx=0 tx=0).stop 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41011b0 msgr2=0x7f17e41973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41011b0 0x7f17e41973a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f17d800d9b0 tx=0x7f17d800dcc0 comp rx=0 tx=0).stop 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 shutdown_connections 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f17cc0a2d60 0x7f17cc0a5220 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17e4102d50 0x7f17e419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17e41011b0 0x7f17e41973a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 --2- 172.21.15.112:0/675675517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17e41023b0 0x7f17e41978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:59.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 >> 172.21.15.112:0/675675517 conn(0x7f17e40fc750 msgr2=0x7f17e41055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:59.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 shutdown_connections 2024-04-24T23:12:59.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:12:59.171+0000 7f17e8d2e700 1 -- 172.21.15.112:0/675675517 wait complete. 2024-04-24T23:12:59.805 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:12:59.805 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T23:12:59.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T23:12:59.843 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T23:12:59.843 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T23:12:59.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T23:13:00.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:59 smithi112 ceph-mon[29186]: pgmap v306: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-04-24T23:13:00.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:59 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/675675517' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:13:00.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:12:59 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/675675517' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:13:00.126 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:59 smithi187 ceph-mon[33292]: pgmap v306: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-04-24T23:13:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:59 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/675675517' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:13:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:12:59 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/675675517' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:13:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:59 smithi150 ceph-mon[32247]: pgmap v306: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2024-04-24T23:13:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:59 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/675675517' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:13:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:12:59 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/675675517' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:13:01.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 -- 172.21.15.112:0/1856156088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 msgr2=0x7f7488107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:01.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 --2- 172.21.15.112:0/1856156088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 0x7f7488107580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f748000b0a0 tx=0x7f748000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 -- 172.21.15.112:0/1856156088 shutdown_connections 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 --2- 172.21.15.112:0/1856156088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7488101af0 0x7f748810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 --2- 172.21.15.112:0/1856156088 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 0x7f7488107580 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 --2- 172.21.15.112:0/1856156088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7488101130 0x7f74881015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 -- 172.21.15.112:0/1856156088 >> 172.21.15.112:0/1856156088 conn(0x7f74880780e0 msgr2=0x7f74880784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 -- 172.21.15.112:0/1856156088 shutdown_connections 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.967+0000 7f748ce66700 1 -- 172.21.15.112:0/1856156088 wait complete. 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f748ce66700 1 Processor -- start 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f748ce66700 1 -- start start 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f748ce66700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7488101130 0x7f7488199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f748ce66700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7488101af0 0x7f7488199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f748ce66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 0x7f74881a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f748ce66700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f74881115d0 con 0x7f74881071a0 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f748ce66700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7488111450 con 0x7f7488101af0 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f748ce66700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7488111750 con 0x7f7488101130 2024-04-24T23:13:01.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f7486d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 0x7f74881a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f7486d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 0x7f74881a4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52774/0 (socket says 172.21.15.112:52774) 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f7486d9d700 1 -- 172.21.15.112:0/1397360054 learned_addr learned my addr 172.21.15.112:0/1397360054 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.968+0000 7f7485d9b700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7488101af0 0x7f7488199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f748659c700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7488101130 0x7f7488199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f7486d9d700 1 -- 172.21.15.112:0/1397360054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7488101130 msgr2=0x7f7488199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f7486d9d700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7488101130 0x7f7488199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f7486d9d700 1 -- 172.21.15.112:0/1397360054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7488101af0 msgr2=0x7f7488199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f7486d9d700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7488101af0 0x7f7488199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f7486d9d700 1 -- 172.21.15.112:0/1397360054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74881a4ed0 con 0x7f74881071a0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f7486d9d700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 0x7f74881a4990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f747800c900 tx=0x7f747800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f747f7fe700 1 -- 172.21.15.112:0/1397360054 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74780078d0 con 0x7f74881071a0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74881a50a0 con 0x7f74881071a0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f747f7fe700 1 -- 172.21.15.112:0/1397360054 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f747800f450 con 0x7f74881071a0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f747f7fe700 1 -- 172.21.15.112:0/1397360054 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478018660 con 0x7f74881071a0 2024-04-24T23:13:01.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.969+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881a5600 con 0x7f74881071a0 2024-04-24T23:13:01.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.970+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7488068f20 con 0x7f74881071a0 2024-04-24T23:13:01.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.971+0000 7f747f7fe700 1 -- 172.21.15.112:0/1397360054 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f747800f600 con 0x7f74881071a0 2024-04-24T23:13:01.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.972+0000 7f747f7fe700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f74740a2d30 0x7f74740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:01.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.972+0000 7f748659c700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f74740a2d30 0x7f74740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:01.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.972+0000 7f747f7fe700 1 -- 172.21.15.112:0/1397360054 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f74780cf1e0 con 0x7f74881071a0 2024-04-24T23:13:01.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.972+0000 7f748659c700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f74740a2d30 0x7f74740a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7480009fd0 tx=0x7f748000b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:01.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:01.976+0000 7f747f7fe700 1 -- 172.21.15.112:0/1397360054 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7478014030 con 0x7f74881071a0 2024-04-24T23:13:02.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:01 smithi112 ceph-mon[29186]: pgmap v307: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-24T23:13:02.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.230+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f748806a4b0 con 0x7f74881071a0 2024-04-24T23:13:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:01 smithi187 ceph-mon[33292]: pgmap v307: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-24T23:13:02.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.234+0000 7f747f7fe700 1 -- 172.21.15.112:0/1397360054 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263208 (secure 0 0 0) 0x7f747809be40 con 0x7f74881071a0 2024-04-24T23:13:02.242 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f74740a2d30 msgr2=0x7f74740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f74740a2d30 0x7f74740a51f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7480009fd0 tx=0x7f748000b7b0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 msgr2=0x7f74881a4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 0x7f74881a4990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f747800c900 tx=0x7f747800cc10 comp rx=0 tx=0).stop 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 shutdown_connections 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f74740a2d30 0x7f74740a51f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74881071a0 0x7f74881a4990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7488101130 0x7f7488199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 --2- 172.21.15.112:0/1397360054 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7488101af0 0x7f7488199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 >> 172.21.15.112:0/1397360054 conn(0x7f74880780e0 msgr2=0x7f74880ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 shutdown_connections 2024-04-24T23:13:02.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:02.237+0000 7f748ce66700 1 -- 172.21.15.112:0/1397360054 wait complete. 2024-04-24T23:13:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:01 smithi150 ceph-mon[32247]: pgmap v307: 129 pgs: 129 active+clean; 628 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-24T23:13:02.972 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":2191305617},{"type":"v1","addr":"172.21.15.112:6801","nonce":2191305617}]},"active_addr":"172.21.15.112:6801/2191305617","active_change":"2024-04-24T23:05:26.702200+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.112:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":182592862}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":4077926577}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1978319668}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":845221192}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":4082045612}]}]} 2024-04-24T23:13:02.977 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T23:13:02.977 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T23:13:02.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:13:03.216 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:03.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:02 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1397360054' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:13:03.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:02 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1397360054' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:13:03.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:02 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1397360054' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:13:04.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:03 smithi187 ceph-mon[33292]: pgmap v308: 129 pgs: 129 active+clean; 628 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.3 KiB/s wr, 22 op/s 2024-04-24T23:13:04.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:03 smithi112 ceph-mon[29186]: pgmap v308: 129 pgs: 129 active+clean; 628 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.3 KiB/s wr, 22 op/s 2024-04-24T23:13:04.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:03 smithi150 ceph-mon[32247]: pgmap v308: 129 pgs: 129 active+clean; 628 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 8.3 KiB/s wr, 22 op/s 2024-04-24T23:13:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.023+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2641232270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 msgr2=0x7f9c74101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.023+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2641232270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 0x7f9c74101230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9c680099c0 tx=0x7f9c68009cd0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2641232270 shutdown_connections 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2641232270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c74107670 0x7f9c7410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2641232270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 0x7f9c74101230 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2641232270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c74101800 0x7f9c74106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2641232270 >> 172.21.15.112:0/2641232270 conn(0x7f9c74076450 msgr2=0x7f9c74076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2641232270 shutdown_connections 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2641232270 wait complete. 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 Processor -- start 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 -- start start 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 0x7f9c7419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.024+0000 7f9c7b03f700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c74101800 0x7f9c7419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c7b03f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c74107670 0x7f9c741987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:05.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c7b03f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741126c0 con 0x7f9c74100e50 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c7b03f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74112540 con 0x7f9c74107670 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c7b03f700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74112840 con 0x7f9c74101800 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 0x7f9c7419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 0x7f9c7419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52792/0 (socket says 172.21.15.112:52792) 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 -- 172.21.15.112:0/2739295447 learned_addr learned my addr 172.21.15.112:0/2739295447 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c73fff700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c74101800 0x7f9c7419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 -- 172.21.15.112:0/2739295447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c74101800 msgr2=0x7f9c7419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c74101800 0x7f9c7419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 -- 172.21.15.112:0/2739295447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c74107670 msgr2=0x7f9c741987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c795dc700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c74107670 0x7f9c741987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c74107670 0x7f9c741987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 -- 172.21.15.112:0/2739295447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c74198f80 con 0x7f9c74100e50 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c73fff700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c74101800 0x7f9c7419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c795dc700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c74107670 0x7f9c741987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:05.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c78ddb700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 0x7f9c7419e620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800b580 tx=0x7f9c68003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:05.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c71ffb700 1 -- 172.21.15.112:0/2739295447 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68027070 con 0x7f9c74100e50 2024-04-24T23:13:05.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c74199210 con 0x7f9c74100e50 2024-04-24T23:13:05.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c71ffb700 1 -- 172.21.15.112:0/2739295447 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9c680041f0 con 0x7f9c74100e50 2024-04-24T23:13:05.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.025+0000 7f9c71ffb700 1 -- 172.21.15.112:0/2739295447 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800f6c0 con 0x7f9c74100e50 2024-04-24T23:13:05.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.026+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c741a5480 con 0x7f9c74100e50 2024-04-24T23:13:05.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.026+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7406a420 con 0x7f9c74100e50 2024-04-24T23:13:05.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.028+0000 7f9c71ffb700 1 -- 172.21.15.112:0/2739295447 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9c6802a070 con 0x7f9c74100e50 2024-04-24T23:13:05.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.028+0000 7f9c71ffb700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c640a2cb0 0x7f9c640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:05.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.028+0000 7f9c73fff700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c640a2cb0 0x7f9c640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:05.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.028+0000 7f9c71ffb700 1 -- 172.21.15.112:0/2739295447 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9c680d02f0 con 0x7f9c74100e50 2024-04-24T23:13:05.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.029+0000 7f9c73fff700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c640a2cb0 0x7f9c640a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005fd0 tx=0x7f9c60005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:05.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.033+0000 7f9c71ffb700 1 -- 172.21.15.112:0/2739295447 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9c6805e740 con 0x7f9c74100e50 2024-04-24T23:13:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.230+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c741a5650 con 0x7f9c74100e50 2024-04-24T23:13:05.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.231+0000 7f9c71ffb700 1 -- 172.21.15.112:0/2739295447 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7f9c68013070 con 0x7f9c74100e50 2024-04-24T23:13:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:13:05.234 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c640a2cb0 msgr2=0x7f9c640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c640a2cb0 0x7f9c640a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9c60005fd0 tx=0x7f9c60005e30 comp rx=0 tx=0).stop 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 msgr2=0x7f9c7419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 0x7f9c7419e620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800b580 tx=0x7f9c68003890 comp rx=0 tx=0).stop 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 shutdown_connections 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c640a2cb0 0x7f9c640a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c74107670 0x7f9c741987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c74100e50 0x7f9c7419e620 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 --2- 172.21.15.112:0/2739295447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c74101800 0x7f9c7419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 >> 172.21.15.112:0/2739295447 conn(0x7f9c74076450 msgr2=0x7f9c740feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 shutdown_connections 2024-04-24T23:13:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:05.234+0000 7f9c7b03f700 1 -- 172.21.15.112:0/2739295447 wait complete. 2024-04-24T23:13:05.937 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:05 smithi112 ceph-mon[29186]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-04-24T23:13:05.938 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:05 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2739295447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:13:05.966 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T23:13:05.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:13:06.205 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:05 smithi187 ceph-mon[33292]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-04-24T23:13:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:05 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2739295447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:13:06.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:05 smithi150 ceph-mon[32247]: pgmap v309: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 24 op/s 2024-04-24T23:13:06.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:05 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2739295447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:13:08.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/1832888484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e4069f40 msgr2=0x7fc7e406a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/1832888484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e4069f40 0x7fc7e406a3a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009990 tx=0x7fc7d8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/1832888484 shutdown_connections 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/1832888484 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e41082b0 0x7fc7e4108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/1832888484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e406a8e0 0x7fc7e410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/1832888484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e4069f40 0x7fc7e406a3a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/1832888484 >> 172.21.15.112:0/1832888484 conn(0x7fc7e40780e0 msgr2=0x7fc7e40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/1832888484 shutdown_connections 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:07.999+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/1832888484 wait complete. 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ebb9e700 1 Processor -- start 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ebb9e700 1 -- start start 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ebb9e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e4069f40 0x7fc7e419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ebb9e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e406a8e0 0x7fc7e419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ebb9e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e41082b0 0x7fc7e4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ebb9e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4112630 con 0x7fc7e41082b0 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ebb9e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e41124b0 con 0x7fc7e406a8e0 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ebb9e700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e41127b0 con 0x7fc7e4069f40 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.000+0000 7fc7ea13b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e41082b0 0x7fc7e4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7e9139700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e406a8e0 0x7fc7e419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7ea13b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e41082b0 0x7fc7e4198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52808/0 (socket says 172.21.15.112:52808) 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7e993a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e4069f40 0x7fc7e419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7ea13b700 1 -- 172.21.15.112:0/2649071489 learned_addr learned my addr 172.21.15.112:0/2649071489 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7ea13b700 1 -- 172.21.15.112:0/2649071489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e4069f40 msgr2=0x7fc7e419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7ea13b700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e4069f40 0x7fc7e419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7ea13b700 1 -- 172.21.15.112:0/2649071489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e406a8e0 msgr2=0x7fc7e419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7ea13b700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e406a8e0 0x7fc7e419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7ea13b700 1 -- 172.21.15.112:0/2649071489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e4199030 con 0x7fc7e41082b0 2024-04-24T23:13:08.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7e9139700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e406a8e0 0x7fc7e419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7e993a700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e4069f40 0x7fc7e419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7ea13b700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e41082b0 0x7fc7e4198850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400ad10 tx=0x7fc7d40074a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.001+0000 7fc7e2ffd700 1 -- 172.21.15.112:0/2649071489 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d401d070 con 0x7fc7e41082b0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.002+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e4199320 con 0x7fc7e41082b0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.002+0000 7fc7e2ffd700 1 -- 172.21.15.112:0/2649071489 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc7d4010530 con 0x7fc7e41082b0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.002+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e4101130 con 0x7fc7e41082b0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.002+0000 7fc7e2ffd700 1 -- 172.21.15.112:0/2649071489 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400fe20 con 0x7fc7e41082b0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.003+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e4103960 con 0x7fc7e41082b0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.004+0000 7fc7e2ffd700 1 -- 172.21.15.112:0/2649071489 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc7d4020070 con 0x7fc7e41082b0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.005+0000 7fc7e2ffd700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc7cc0a2cb0 0x7fc7cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.005+0000 7fc7e2ffd700 1 -- 172.21.15.112:0/2649071489 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc7d40d2cc0 con 0x7fc7e41082b0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.005+0000 7fc7e993a700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc7cc0a2cb0 0x7fc7cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.005+0000 7fc7e993a700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc7cc0a2cb0 0x7fc7cc0a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0060b0 tx=0x7fc7dc00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.009+0000 7fc7e2ffd700 1 -- 172.21.15.112:0/2649071489 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc7d4027020 con 0x7fc7e41082b0 2024-04-24T23:13:08.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:07 smithi112 ceph-mon[29186]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-24T23:13:08.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.208+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7e4199d80 con 0x7fc7e41082b0 2024-04-24T23:13:08.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.209+0000 7fc7e2ffd700 1 -- 172.21.15.112:0/2649071489 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24187 (secure 0 0 0) 0x7fc7d409f880 con 0x7fc7e41082b0 2024-04-24T23:13:08.212 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:13:08.212 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":90,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:12:32.364231+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:13:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.211+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc7cc0a2cb0 msgr2=0x7fc7cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.211+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc7cc0a2cb0 0x7fc7cc0a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0060b0 tx=0x7fc7dc00c040 comp rx=0 tx=0).stop 2024-04-24T23:13:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.211+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e41082b0 msgr2=0x7fc7e4198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.211+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e41082b0 0x7fc7e4198850 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400ad10 tx=0x7fc7d40074a0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.212+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 shutdown_connections 2024-04-24T23:13:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.212+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc7cc0a2cb0 0x7fc7cc0a5170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.212+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7e41082b0 0x7fc7e4198850 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.212+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7e4069f40 0x7fc7e419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.212+0000 7fc7ebb9e700 1 --2- 172.21.15.112:0/2649071489 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc7e406a8e0 0x7fc7e419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.212+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 >> 172.21.15.112:0/2649071489 conn(0x7fc7e40780e0 msgr2=0x7fc7e40ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.212+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 shutdown_connections 2024-04-24T23:13:08.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:08.212+0000 7fc7ebb9e700 1 -- 172.21.15.112:0/2649071489 wait complete. 2024-04-24T23:13:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:07 smithi187 ceph-mon[33292]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-24T23:13:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:07 smithi150 ceph-mon[32247]: pgmap v310: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 25 op/s 2024-04-24T23:13:09.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T23:13:09.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T23:13:09.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T23:13:09.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T23:13:09.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T23:13:09.036 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T23:13:09.036 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-24T23:13:09.036 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-24T23:13:09.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-24T23:13:09.037 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-24T23:13:09.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-24T23:13:09.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-24T23:13:09.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:08 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2649071489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:13:09.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:08 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2649071489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:13:09.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:08 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2649071489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:13:09.626 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.023 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:09 smithi112 ceph-mon[29186]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-04-24T23:13:10.219 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:09 smithi150 ceph-mon[32247]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-04-24T23:13:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:09 smithi187 ceph-mon[33292]: pgmap v311: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-04-24T23:13:10.519 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.520 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.521 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.523 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.528 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.528 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.531 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.532 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:10.534 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:11.700 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:12.056 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:11 smithi112 ceph-mon[29186]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-24T23:13:12.062 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:12.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:11 smithi187 ceph-mon[33292]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-24T23:13:12.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:11 smithi150 ceph-mon[32247]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-04-24T23:13:14.015 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:13 smithi112 ceph-mon[29186]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-24T23:13:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:13 smithi187 ceph-mon[33292]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-24T23:13:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:13 smithi150 ceph-mon[32247]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-24T23:13:16.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:15 smithi112 ceph-mon[29186]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-24T23:13:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:15 smithi187 ceph-mon[33292]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-24T23:13:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:15 smithi150 ceph-mon[32247]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-24T23:13:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:17 smithi187 ceph-mon[33292]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:17 smithi112 ceph-mon[29186]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:17 smithi150 ceph-mon[32247]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:18.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 -- 172.21.15.112:0/1578891572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 msgr2=0x7f1418105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 --2- 172.21.15.112:0/1578891572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 0x7f1418105d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f141000b0d0 tx=0x7f141000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 -- 172.21.15.112:0/1578891572 shutdown_connections 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 --2- 172.21.15.112:0/1578891572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1418106450 0x7f141810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 --2- 172.21.15.112:0/1578891572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14181013c0 0x7f14181017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 --2- 172.21.15.112:0/1578891572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 0x7f1418105d70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 -- 172.21.15.112:0/1578891572 >> 172.21.15.112:0/1578891572 conn(0x7f1418078100 msgr2=0x7f1418078510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 -- 172.21.15.112:0/1578891572 shutdown_connections 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.785+0000 7f141e804700 1 -- 172.21.15.112:0/1578891572 wait complete. 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141e804700 1 Processor -- start 2024-04-24T23:13:18.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141e804700 1 -- start start 2024-04-24T23:13:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141e804700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14181013c0 0x7f1418199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:18.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141e804700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 0x7f1418199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141e804700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1418106450 0x7f14181a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141e804700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1418111570 con 0x7f1418101d70 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141e804700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f14181113f0 con 0x7f1418106450 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141e804700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14181116f0 con 0x7f14181013c0 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f14177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 0x7f1418199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f14177fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 0x7f1418199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51170/0 (socket says 172.21.15.112:51170) 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f1417fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14181013c0 0x7f1418199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f14177fe700 1 -- 172.21.15.112:0/3613882659 learned_addr learned my addr 172.21.15.112:0/3613882659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.786+0000 7f141cda1700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1418106450 0x7f14181a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14177fe700 1 -- 172.21.15.112:0/3613882659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14181013c0 msgr2=0x7f1418199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14177fe700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14181013c0 0x7f1418199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14177fe700 1 -- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1418106450 msgr2=0x7f14181a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14177fe700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1418106450 0x7f14181a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14177fe700 1 -- 172.21.15.112:0/3613882659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14181a4f10 con 0x7f1418101d70 2024-04-24T23:13:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f141cda1700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1418106450 0x7f14181a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14177fe700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 0x7f1418199e90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1410009bc0 tx=0x7f1410007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410003c00 con 0x7f1418101d70 2024-04-24T23:13:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14181a50e0 con 0x7f1418101d70 2024-04-24T23:13:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1410022070 con 0x7f1418101d70 2024-04-24T23:13:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141001b7b0 con 0x7f1418101d70 2024-04-24T23:13:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.787+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14181a5540 con 0x7f1418101d70 2024-04-24T23:13:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.788+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1418062940 con 0x7f1418101d70 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.789+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f141001da00 con 0x7f1418101d70 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.790+0000 7f14157fa700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14000a2cb0 0x7f14000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.790+0000 7f1417fff700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14000a2cb0 0x7f14000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.790+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f141000f020 con 0x7f1418101d70 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.790+0000 7f14157fa700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] conn(0x7f14000ad970 0x7f14000afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.790+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 --> [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f141000cbc0 con 0x7f14000ad970 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.790+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f14100d07c0 con 0x7f1418101d70 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.790+0000 7f1417fff700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14000a2cb0 0x7f14000a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1408004630 tx=0x7f140800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.790+0000 7f141cda1700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] conn(0x7f14000ad970 0x7f14000afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.791+0000 7f141cda1700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] conn(0x7f14000ad970 0x7f14000afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.792+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 <== osd.7 v2:172.21.15.150:6816/591791108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f141000cbc0 con 0x7f14000ad970 2024-04-24T23:13:18.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.808+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 --> [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1418062dc0 con 0x7f14000ad970 2024-04-24T23:13:18.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.808+0000 7f14157fa700 1 -- 172.21.15.112:0/3613882659 <== osd.7 v2:172.21.15.150:6816/591791108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1418062dc0 con 0x7f14000ad970 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] conn(0x7f14000ad970 msgr2=0x7f14000afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] conn(0x7f14000ad970 0x7f14000afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14000a2cb0 msgr2=0x7f14000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14000a2cb0 0x7f14000a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1408004630 tx=0x7f140800b3c0 comp rx=0 tx=0).stop 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 msgr2=0x7f1418199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 --2- 172.21.15.112:0/3613882659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418101d70 0x7f1418199e90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1410009bc0 tx=0x7f1410007a20 comp rx=0 tx=0).stop 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141cda1700 1 -- 172.21.15.112:0/3613882659 reap_dead start 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 shutdown_connections 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 >> 172.21.15.112:0/3613882659 conn(0x7f1418078100 msgr2=0x7f14180ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:18.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 shutdown_connections 2024-04-24T23:13:18.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:18.809+0000 7f141e804700 1 -- 172.21.15.112:0/3613882659 wait complete. 2024-04-24T23:13:20.190 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:19 smithi112 ceph-mon[29186]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:19 smithi187 ceph-mon[33292]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:19 smithi150 ceph-mon[32247]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:21.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 -- 172.21.15.112:0/727977416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 msgr2=0x7fae0c1075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 --2- 172.21.15.112:0/727977416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 0x7fae0c1075b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009990 tx=0x7fadfc009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 -- 172.21.15.112:0/727977416 shutdown_connections 2024-04-24T23:13:21.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 --2- 172.21.15.112:0/727977416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae0c101b20 0x7fae0c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 --2- 172.21.15.112:0/727977416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 0x7fae0c1075b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 --2- 172.21.15.112:0/727977416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae0c101160 0x7fae0c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 -- 172.21.15.112:0/727977416 >> 172.21.15.112:0/727977416 conn(0x7fae0c078170 msgr2=0x7fae0c078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 -- 172.21.15.112:0/727977416 shutdown_connections 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.120+0000 7fae10fec700 1 -- 172.21.15.112:0/727977416 wait complete. 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae10fec700 1 Processor -- start 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae10fec700 1 -- start start 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae10fec700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae0c101160 0x7fae0c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae10fec700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae0c101b20 0x7fae0c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae10fec700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 0x7fae0c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae10fec700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c1115a0 con 0x7fae0c1071d0 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae10fec700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c111420 con 0x7fae0c101160 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae10fec700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c111720 con 0x7fae0c101b20 2024-04-24T23:13:21.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae0ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 0x7fae0c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae0ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 0x7fae0c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56476/0 (socket says 172.21.15.112:56476) 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae0ad9d700 1 -- 172.21.15.112:0/358500571 learned_addr learned my addr 172.21.15.112:0/358500571 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae0a59c700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae0c101160 0x7fae0c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.121+0000 7fae03fff700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae0c101b20 0x7fae0c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae0ad9d700 1 -- 172.21.15.112:0/358500571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae0c101b20 msgr2=0x7fae0c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae0ad9d700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae0c101b20 0x7fae0c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae0ad9d700 1 -- 172.21.15.112:0/358500571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae0c101160 msgr2=0x7fae0c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae0ad9d700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fae0c101160 0x7fae0c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae0ad9d700 1 -- 172.21.15.112:0/358500571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c199050 con 0x7fae0c1071d0 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae0ad9d700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 0x7fae0c1988f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae0400ada0 tx=0x7fae0400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04017020 con 0x7fae0c1071d0 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c199340 con 0x7fae0c1071d0 2024-04-24T23:13:21.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fae0400a0d0 con 0x7fae0c1071d0 2024-04-24T23:13:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c1a5600 con 0x7fae0c1071d0 2024-04-24T23:13:21.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.122+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04005440 con 0x7fae0c1071d0 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.123+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae0c06ac50 con 0x7fae0c1071d0 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.124+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fae0400c980 con 0x7fae0c1071d0 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.125+0000 7fae037fe700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fadf80a2cb0 0x7fadf80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.125+0000 7fae0a59c700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fadf80a2cb0 0x7fadf80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.125+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fae040cf4e0 con 0x7fae0c1071d0 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.125+0000 7fae037fe700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] conn(0x7fadf80ad970 0x7fadf80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.125+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 --> [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae0401ee50 con 0x7fadf80ad970 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.125+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fae040cf930 con 0x7fae0c1071d0 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.125+0000 7fae0a59c700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fadf80a2cb0 0x7fadf80a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00b5c0 tx=0x7fadfc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:21.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.125+0000 7fae03fff700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] conn(0x7fadf80ad970 0x7fadf80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.126+0000 7fae03fff700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] conn(0x7fadf80ad970 0x7fadf80afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:21.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.127+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 <== osd.1 v2:172.21.15.150:6800/459871910 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae0401ee50 con 0x7fadf80ad970 2024-04-24T23:13:21.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.143+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 --> [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fae0c102880 con 0x7fadf80ad970 2024-04-24T23:13:21.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae037fe700 1 -- 172.21.15.112:0/358500571 <== osd.1 v2:172.21.15.150:6800/459871910 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fae0c102880 con 0x7fadf80ad970 2024-04-24T23:13:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 >> [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] conn(0x7fadf80ad970 msgr2=0x7fadf80afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] conn(0x7fadf80ad970 0x7fadf80afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fadf80a2cb0 msgr2=0x7fadf80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fadf80a2cb0 0x7fadf80a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00b5c0 tx=0x7fadfc005960 comp rx=0 tx=0).stop 2024-04-24T23:13:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 msgr2=0x7fae0c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 --2- 172.21.15.112:0/358500571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae0c1071d0 0x7fae0c1988f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae0400ada0 tx=0x7fae0400c330 comp rx=0 tx=0).stop 2024-04-24T23:13:21.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae0ad9d700 1 -- 172.21.15.112:0/358500571 reap_dead start 2024-04-24T23:13:21.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 shutdown_connections 2024-04-24T23:13:21.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 >> 172.21.15.112:0/358500571 conn(0x7fae0c078170 msgr2=0x7fae0c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:21.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 shutdown_connections 2024-04-24T23:13:21.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.144+0000 7fae10fec700 1 -- 172.21.15.112:0/358500571 wait complete. 2024-04-24T23:13:21.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 -- 172.21.15.112:0/42808227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 msgr2=0x7f6618106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 --2- 172.21.15.112:0/42808227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 0x7f6618106ec0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6608009990 tx=0x7f6608009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 -- 172.21.15.112:0/42808227 shutdown_connections 2024-04-24T23:13:21.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 --2- 172.21.15.112:0/42808227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66181075a0 0x7f661810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 --2- 172.21.15.112:0/42808227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618102540 0x7f6618102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 --2- 172.21.15.112:0/42808227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 0x7f6618106ec0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 -- 172.21.15.112:0/42808227 >> 172.21.15.112:0/42808227 conn(0x7f6618079310 msgr2=0x7f6618079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:21.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 -- 172.21.15.112:0/42808227 shutdown_connections 2024-04-24T23:13:21.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.686+0000 7f661fae3700 1 -- 172.21.15.112:0/42808227 wait complete. 2024-04-24T23:13:21.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661fae3700 1 Processor -- start 2024-04-24T23:13:21.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661fae3700 1 -- start start 2024-04-24T23:13:21.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661fae3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618102540 0x7f6618074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661fae3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 0x7f6618074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661fae3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66181075a0 0x7f6618074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661fae3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66181126e0 con 0x7f6618102ef0 2024-04-24T23:13:21.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661fae3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f6618112560 con 0x7f66181075a0 2024-04-24T23:13:21.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661fae3700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6618112860 con 0x7f6618102540 2024-04-24T23:13:21.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661d07e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 0x7f6618074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661d07e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 0x7f6618074a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56492/0 (socket says 172.21.15.112:56492) 2024-04-24T23:13:21.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.687+0000 7f661d07e700 1 -- 172.21.15.112:0/3433568446 learned_addr learned my addr 172.21.15.112:0/3433568446 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:21.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661e080700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66181075a0 0x7f6618074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661d87f700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618102540 0x7f6618074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661d07e700 1 -- 172.21.15.112:0/3433568446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618102540 msgr2=0x7f6618074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661d07e700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6618102540 0x7f6618074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661d07e700 1 -- 172.21.15.112:0/3433568446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66181075a0 msgr2=0x7f6618074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661d07e700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66181075a0 0x7f6618074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661d07e700 1 -- 172.21.15.112:0/3433568446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66181a7020 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661d07e700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 0x7f6618074a70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6608000c00 tx=0x7f660800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608027070 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66181a72b0 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f660800fce0 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66080173f0 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.688+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66181a7700 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.689+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6618062940 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.690+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f660802a070 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.691+0000 7f660effd700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f66040a2cb0 0x7f66040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.691+0000 7f661d87f700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f66040a2cb0 0x7f66040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.691+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f66080d0550 con 0x7f6618102ef0 2024-04-24T23:13:21.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.691+0000 7f660effd700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] conn(0x7f66040ad970 0x7f66040afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.691+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 --> [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6608019e80 con 0x7f66040ad970 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.691+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f66080d09a0 con 0x7f6618102ef0 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.691+0000 7f661d87f700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f66040a2cb0 0x7f66040a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6614004630 tx=0x7f661400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.692+0000 7f661e080700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] conn(0x7f66040ad970 0x7f66040afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.692+0000 7f661e080700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] conn(0x7f66040ad970 0x7f66040afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.693+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 <== osd.4 v2:172.21.15.150:6808/3997288444 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6608019e80 con 0x7f66040ad970 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.709+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 --> [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6618062dc0 con 0x7f66040ad970 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.709+0000 7f660effd700 1 -- 172.21.15.112:0/3433568446 <== osd.4 v2:172.21.15.150:6808/3997288444 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6618062dc0 con 0x7f66040ad970 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 >> [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] conn(0x7f66040ad970 msgr2=0x7f66040afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] conn(0x7f66040ad970 0x7f66040afdb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f66040a2cb0 msgr2=0x7f66040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f66040a2cb0 0x7f66040a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6614004630 tx=0x7f661400b3c0 comp rx=0 tx=0).stop 2024-04-24T23:13:21.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 msgr2=0x7f6618074a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:21.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 --2- 172.21.15.112:0/3433568446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6618102ef0 0x7f6618074a70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6608000c00 tx=0x7f660800f460 comp rx=0 tx=0).stop 2024-04-24T23:13:21.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661e080700 1 -- 172.21.15.112:0/3433568446 reap_dead start 2024-04-24T23:13:21.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 shutdown_connections 2024-04-24T23:13:21.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 >> 172.21.15.112:0/3433568446 conn(0x7f6618079310 msgr2=0x7f6618101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:21.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 shutdown_connections 2024-04-24T23:13:21.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:21.710+0000 7f661fae3700 1 -- 172.21.15.112:0/3433568446 wait complete. 2024-04-24T23:13:22.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:21 smithi112 ceph-mon[29186]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:21 smithi187 ceph-mon[33292]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:21 smithi150 ceph-mon[32247]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:13:23.374 INFO:teuthology.orchestra.run.smithi112.stdout:141733920828 2024-04-24T23:13:23.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 -- 172.21.15.112:0/3207255437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 msgr2=0x7fa328106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 --2- 172.21.15.112:0/3207255437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 0x7fa328106ec0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009990 tx=0x7fa31c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 -- 172.21.15.112:0/3207255437 shutdown_connections 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 --2- 172.21.15.112:0/3207255437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3281075a0 0x7fa32810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 --2- 172.21.15.112:0/3207255437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa328102540 0x7fa328102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 --2- 172.21.15.112:0/3207255437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 0x7fa328106ec0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 -- 172.21.15.112:0/3207255437 >> 172.21.15.112:0/3207255437 conn(0x7fa3280792c0 msgr2=0x7fa3280796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 -- 172.21.15.112:0/3207255437 shutdown_connections 2024-04-24T23:13:23.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 -- 172.21.15.112:0/3207255437 wait complete. 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.566+0000 7fa32e065700 1 Processor -- start 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa32e065700 1 -- start start 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa32e065700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa328102540 0x7fa32819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa32e065700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 0x7fa32819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa32e065700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3281075a0 0x7fa328198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa32e065700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa328112690 con 0x7fa328102ef0 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa32e065700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa328112510 con 0x7fa3281075a0 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa32e065700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa328112810 con 0x7fa328102540 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa326ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 0x7fa32819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa326ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 0x7fa32819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56516/0 (socket says 172.21.15.112:56516) 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa326ffd700 1 -- 172.21.15.112:0/1747484345 learned_addr learned my addr 172.21.15.112:0/1747484345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa3277fe700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa328102540 0x7fa32819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa327fff700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3281075a0 0x7fa328198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa326ffd700 1 -- 172.21.15.112:0/1747484345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa328102540 msgr2=0x7fa32819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa326ffd700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa328102540 0x7fa32819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa326ffd700 1 -- 172.21.15.112:0/1747484345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3281075a0 msgr2=0x7fa328198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa326ffd700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3281075a0 0x7fa328198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa326ffd700 1 -- 172.21.15.112:0/1747484345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa328199070 con 0x7fa328102ef0 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.567+0000 7fa327fff700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3281075a0 0x7fa328198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.568+0000 7fa326ffd700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 0x7fa32819ec90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00b840 tx=0x7fa31c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.568+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c00fac0 con 0x7fa328102ef0 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.568+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa328199300 con 0x7fa328102ef0 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.568+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa31c004d30 con 0x7fa328102ef0 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.568+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c022470 con 0x7fa328102ef0 2024-04-24T23:13:23.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.568+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3281a5620 con 0x7fa328102ef0 2024-04-24T23:13:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.569+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3280629c0 con 0x7fa328102ef0 2024-04-24T23:13:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.570+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa31c029020 con 0x7fa328102ef0 2024-04-24T23:13:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.571+0000 7fa324ff9700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3100a2cb0 0x7fa3100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.571+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa31c0d11f0 con 0x7fa328102ef0 2024-04-24T23:13:23.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.571+0000 7fa3277fe700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3100a2cb0 0x7fa3100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.571+0000 7fa324ff9700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] conn(0x7fa3100ad970 0x7fa3100afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.571+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 --> [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa31c00eea0 con 0x7fa3100ad970 2024-04-24T23:13:23.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.571+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa31c0d16d0 con 0x7fa328102ef0 2024-04-24T23:13:23.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.572+0000 7fa3277fe700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3100a2cb0 0x7fa3100a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa328199a50 tx=0x7fa3180073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:23.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.572+0000 7fa327fff700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] conn(0x7fa3100ad970 0x7fa3100afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.572+0000 7fa327fff700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] conn(0x7fa3100ad970 0x7fa3100afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:23.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.573+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 <== osd.2 v2:172.21.15.187:6800/3027054636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa31c00eea0 con 0x7fa3100ad970 2024-04-24T23:13:23.591 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.590+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 --> [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa328062dc0 con 0x7fa3100ad970 2024-04-24T23:13:23.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.590+0000 7fa324ff9700 1 -- 172.21.15.112:0/1747484345 <== osd.2 v2:172.21.15.187:6800/3027054636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa328062dc0 con 0x7fa3100ad970 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 >> [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] conn(0x7fa3100ad970 msgr2=0x7fa3100afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] conn(0x7fa3100ad970 0x7fa3100afdb0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3100a2cb0 msgr2=0x7fa3100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3100a2cb0 0x7fa3100a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa328199a50 tx=0x7fa3180073d0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 msgr2=0x7fa32819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 --2- 172.21.15.112:0/1747484345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa328102ef0 0x7fa32819ec90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00b840 tx=0x7fa31c00b870 comp rx=0 tx=0).stop 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa327fff700 1 -- 172.21.15.112:0/1747484345 reap_dead start 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 shutdown_connections 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 >> 172.21.15.112:0/1747484345 conn(0x7fa3280792c0 msgr2=0x7fa328100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 shutdown_connections 2024-04-24T23:13:23.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.591+0000 7fa32e065700 1 -- 172.21.15.112:0/1747484345 wait complete. 2024-04-24T23:13:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 -- 172.21.15.112:0/305579678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5598101130 msgr2=0x7f55981015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 --2- 172.21.15.112:0/305579678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5598101130 0x7f55981015b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5588009990 tx=0x7f5588009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 -- 172.21.15.112:0/305579678 shutdown_connections 2024-04-24T23:13:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 --2- 172.21.15.112:0/305579678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55981071a0 0x7f5598107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 --2- 172.21.15.112:0/305579678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598101af0 0x7f559810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 --2- 172.21.15.112:0/305579678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5598101130 0x7f55981015b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 -- 172.21.15.112:0/305579678 >> 172.21.15.112:0/305579678 conn(0x7f55980780e0 msgr2=0x7f55980784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 -- 172.21.15.112:0/305579678 shutdown_connections 2024-04-24T23:13:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.766+0000 7f55a07a4700 1 -- 172.21.15.112:0/305579678 wait complete. 2024-04-24T23:13:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f55a07a4700 1 Processor -- start 2024-04-24T23:13:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f55a07a4700 1 -- start start 2024-04-24T23:13:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f55a07a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598101130 0x7f5598199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f55a07a4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598101af0 0x7f5598199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f55a07a4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55981071a0 0x7f55981a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f55a07a4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5598111550 con 0x7f55981071a0 2024-04-24T23:13:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f55a07a4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f55981113d0 con 0x7f5598101130 2024-04-24T23:13:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f55a07a4700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55981116d0 con 0x7f5598101af0 2024-04-24T23:13:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f559ed41700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55981071a0 0x7f55981a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f559ed41700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55981071a0 0x7f55981a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56540/0 (socket says 172.21.15.112:56540) 2024-04-24T23:13:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f559ed41700 1 -- 172.21.15.112:0/1137496006 learned_addr learned my addr 172.21.15.112:0/1137496006 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f559e540700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598101130 0x7f5598199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.767+0000 7f559dd3f700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598101af0 0x7f5598199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f559ed41700 1 -- 172.21.15.112:0/1137496006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598101af0 msgr2=0x7f5598199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f559ed41700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5598101af0 0x7f5598199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f559ed41700 1 -- 172.21.15.112:0/1137496006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598101130 msgr2=0x7f5598199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f559ed41700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5598101130 0x7f5598199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f559ed41700 1 -- 172.21.15.112:0/1137496006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55981a4dd0 con 0x7f55981071a0 2024-04-24T23:13:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f559ed41700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55981071a0 0x7f55981a4890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5590007d70 tx=0x7f559000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590016020 con 0x7f55981071a0 2024-04-24T23:13:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f559000ad40 con 0x7f55981071a0 2024-04-24T23:13:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55900116b0 con 0x7f55981071a0 2024-04-24T23:13:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55981a4fa0 con 0x7f55981071a0 2024-04-24T23:13:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.768+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55981a54b0 con 0x7f55981071a0 2024-04-24T23:13:23.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.769+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f55981003e0 con 0x7f55981071a0 2024-04-24T23:13:23.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.770+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5590019020 con 0x7f55981071a0 2024-04-24T23:13:23.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.771+0000 7f558f7fe700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f55840a2cb0 0x7f55840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.771+0000 7f559e540700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f55840a2cb0 0x7f55840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.771+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f55900cf7f0 con 0x7f55981071a0 2024-04-24T23:13:23.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.771+0000 7f558f7fe700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] conn(0x7f55840ad970 0x7f55840afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.771+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 --> [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5590022e50 con 0x7f55840ad970 2024-04-24T23:13:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.771+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f55900cfc40 con 0x7f55981071a0 2024-04-24T23:13:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.771+0000 7f559dd3f700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] conn(0x7f55840ad970 0x7f55840afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.771+0000 7f559e540700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f55840a2cb0 0x7f55840a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f559400abd0 tx=0x7f559400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.772+0000 7f559dd3f700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] conn(0x7f55840ad970 0x7f55840afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:23.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.772+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 <== osd.6 v2:172.21.15.112:6818/1354124649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5590022e50 con 0x7f55840ad970 2024-04-24T23:13:23.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 --> [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5598102850 con 0x7f55840ad970 2024-04-24T23:13:23.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f558f7fe700 1 -- 172.21.15.112:0/1137496006 <== osd.6 v2:172.21.15.112:6818/1354124649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5598102850 con 0x7f55840ad970 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] conn(0x7f55840ad970 msgr2=0x7f55840afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] conn(0x7f55840ad970 0x7f55840afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f55840a2cb0 msgr2=0x7f55840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f55840a2cb0 0x7f55840a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f559400abd0 tx=0x7f559400a380 comp rx=0 tx=0).stop 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55981071a0 msgr2=0x7f55981a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 --2- 172.21.15.112:0/1137496006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55981071a0 0x7f55981a4890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5590007d70 tx=0x7f559000a340 comp rx=0 tx=0).stop 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f559ed41700 1 -- 172.21.15.112:0/1137496006 reap_dead start 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 shutdown_connections 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 >> 172.21.15.112:0/1137496006 conn(0x7f55980780e0 msgr2=0x7f55980ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 shutdown_connections 2024-04-24T23:13:23.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:23.789+0000 7f55a07a4700 1 -- 172.21.15.112:0/1137496006 wait complete. 2024-04-24T23:13:24.229 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:23 smithi112 ceph-mon[29186]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-24T23:13:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:23 smithi187 ceph-mon[33292]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-24T23:13:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:23 smithi150 ceph-mon[32247]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-24T23:13:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 -- 172.21.15.112:0/1255316807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101ae0 msgr2=0x7fcfa010dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 --2- 172.21.15.112:0/1255316807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101ae0 0x7fcfa010dfd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800b0a0 tx=0x7fcf9800b3b0 comp rx=0 tx=0).stop 2024-04-24T23:13:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 -- 172.21.15.112:0/1255316807 shutdown_connections 2024-04-24T23:13:25.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 --2- 172.21.15.112:0/1255316807 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101ae0 0x7fcfa010dfd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 --2- 172.21.15.112:0/1255316807 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa0107190 0x7fcfa0107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 --2- 172.21.15.112:0/1255316807 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfa0101120 0x7fcfa01015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 -- 172.21.15.112:0/1255316807 >> 172.21.15.112:0/1255316807 conn(0x7fcfa00780a0 msgr2=0x7fcfa00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 -- 172.21.15.112:0/1255316807 shutdown_connections 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 -- 172.21.15.112:0/1255316807 wait complete. 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 Processor -- start 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.850+0000 7fcfa6292700 1 -- start start 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcfa6292700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101120 0x7fcfa0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcfa6292700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa0101ae0 0x7fcfa0199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcfa6292700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfa0107190 0x7fcfa01a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcfa6292700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa0111560 con 0x7fcfa0101120 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcfa6292700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa01113e0 con 0x7fcfa0107190 2024-04-24T23:13:25.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcfa6292700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa01116e0 con 0x7fcfa0101ae0 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101120 0x7fcfa0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101120 0x7fcfa0199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56560/0 (socket says 172.21.15.112:56560) 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 -- 172.21.15.112:0/259497822 learned_addr learned my addr 172.21.15.112:0/259497822 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf97fff700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa0101ae0 0x7fcfa0199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcfa482f700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfa0107190 0x7fcfa01a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 -- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa0101ae0 msgr2=0x7fcfa0199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa0101ae0 0x7fcfa0199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 -- 172.21.15.112:0/259497822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfa0107190 msgr2=0x7fcfa01a4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfa0107190 0x7fcfa01a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 -- 172.21.15.112:0/259497822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa01a4ec0 con 0x7fcfa0101120 2024-04-24T23:13:25.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf97fff700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa0101ae0 0x7fcfa0199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9ffff700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101120 0x7fcfa0199900 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000d980 tx=0x7fcf9000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90017070 con 0x7fcfa0101120 2024-04-24T23:13:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fcf90004d30 con 0x7fcfa0101120 2024-04-24T23:13:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa01a50f0 con 0x7fcfa0101120 2024-04-24T23:13:25.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.851+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90005610 con 0x7fcfa0101120 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.852+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa01a55b0 con 0x7fcfa0101120 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.853+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcfa0062940 con 0x7fcfa0101120 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.854+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcf90004750 con 0x7fcfa0101120 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.854+0000 7fcf9dffb700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcf8c0a2cb0 0x7fcf8c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.854+0000 7fcf97fff700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcf8c0a2cb0 0x7fcf8c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.854+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fcf900ceff0 con 0x7fcfa0101120 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.854+0000 7fcf9dffb700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] conn(0x7fcf8c0ad970 0x7fcf8c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.854+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 --> [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf90014ec0 con 0x7fcf8c0ad970 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.854+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fcf900d6050 con 0x7fcfa0101120 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.855+0000 7fcf97fff700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcf8c0a2cb0 0x7fcf8c0a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005d90 tx=0x7fcf88005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.855+0000 7fcfa482f700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] conn(0x7fcf8c0ad970 0x7fcf8c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.855+0000 7fcfa482f700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] conn(0x7fcf8c0ad970 0x7fcf8c0afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:25.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.856+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 <== osd.11 v2:172.21.15.187:6824/3702998666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf90014ec0 con 0x7fcf8c0ad970 2024-04-24T23:13:25.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.872+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 --> [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcfa0062d70 con 0x7fcf8c0ad970 2024-04-24T23:13:25.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.873+0000 7fcf9dffb700 1 -- 172.21.15.112:0/259497822 <== osd.11 v2:172.21.15.187:6824/3702998666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcfa0062d70 con 0x7fcf8c0ad970 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] conn(0x7fcf8c0ad970 msgr2=0x7fcf8c0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] conn(0x7fcf8c0ad970 0x7fcf8c0afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcf8c0a2cb0 msgr2=0x7fcf8c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fcf8c0a2cb0 0x7fcf8c0a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf88005d90 tx=0x7fcf88005d00 comp rx=0 tx=0).stop 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101120 msgr2=0x7fcfa0199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 --2- 172.21.15.112:0/259497822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0101120 0x7fcfa0199900 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000d980 tx=0x7fcf9000dd40 comp rx=0 tx=0).stop 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa482f700 1 -- 172.21.15.112:0/259497822 reap_dead start 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 shutdown_connections 2024-04-24T23:13:25.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 >> 172.21.15.112:0/259497822 conn(0x7fcfa00780a0 msgr2=0x7fcfa00ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:25.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 shutdown_connections 2024-04-24T23:13:25.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:25.874+0000 7fcfa6292700 1 -- 172.21.15.112:0/259497822 wait complete. 2024-04-24T23:13:26.126 INFO:teuthology.orchestra.run.smithi112.stdout:55834574922 2024-04-24T23:13:26.127 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T23:13:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:25 smithi187 ceph-mon[33292]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T23:13:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:25 smithi112 ceph-mon[29186]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T23:13:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:25 smithi150 ceph-mon[32247]: pgmap v319: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T23:13:26.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:26.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-04-24T23:13:26.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-24T23:13:26.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-24T23:13:26.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T23:13:26.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-04-24T23:13:26.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-24T23:13:26.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T23:13:26.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-24T23:13:26.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-24T23:13:26.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:26 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-24T23:13:27.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:27.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-04-24T23:13:27.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-24T23:13:27.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-24T23:13:27.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T23:13:27.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-04-24T23:13:27.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-24T23:13:27.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T23:13:27.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-24T23:13:27.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-24T23:13:27.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:26 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-24T23:13:27.219 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:27.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-04-24T23:13:27.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-24T23:13:27.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-24T23:13:27.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-24T23:13:27.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-04-24T23:13:27.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-04-24T23:13:27.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-24T23:13:27.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-04-24T23:13:27.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-04-24T23:13:27.221 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:26 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-24T23:13:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:13:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-04-24T23:13:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-24T23:13:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-24T23:13:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T23:13:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-04-24T23:13:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-24T23:13:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T23:13:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-24T23:13:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-24T23:13:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-24T23:13:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:13:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-24T23:13:28.253 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:13:28.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.254 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.277 INFO:teuthology.orchestra.run.smithi112.stdout:347892351001 2024-04-24T23:13:28.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: pgmap v320: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-24T23:13:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-04-24T23:13:28.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-04-24T23:13:28.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-24T23:13:28.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: osdmap e91: 12 total, 12 up, 12 in 2024-04-24T23:13:28.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.319 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 -- 172.21.15.112:0/761589130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 msgr2=0x7f8f3c10f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 --2- 172.21.15.112:0/761589130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 0x7f8f3c10f130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b0a0 tx=0x7f8f3400b3b0 comp rx=0 tx=0).stop 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 -- 172.21.15.112:0/761589130 shutdown_connections 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 --2- 172.21.15.112:0/761589130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 0x7f8f3c10f130 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 --2- 172.21.15.112:0/761589130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3c1082f0 0x7f8f3c1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 --2- 172.21.15.112:0/761589130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3c1022b0 0x7f8f3c102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 -- 172.21.15.112:0/761589130 >> 172.21.15.112:0/761589130 conn(0x7f8f3c079300 msgr2=0x7f8f3c079710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 -- 172.21.15.112:0/761589130 shutdown_connections 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.492+0000 7f8f43e68700 1 -- 172.21.15.112:0/761589130 wait complete. 2024-04-24T23:13:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f43e68700 1 Processor -- start 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f43e68700 1 -- start start 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f43e68700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3c1022b0 0x7f8f3c1a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f43e68700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 0x7f8f3c1a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f43e68700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3c1082f0 0x7f8f3c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f43e68700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c112680 con 0x7f8f3c102c50 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f43e68700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c112500 con 0x7f8f3c1082f0 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f43e68700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c112800 con 0x7f8f3c1022b0 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f41403700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 0x7f8f3c1a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f41403700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 0x7f8f3c1a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56576/0 (socket says 172.21.15.112:56576) 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f41403700 1 -- 172.21.15.112:0/377653119 learned_addr learned my addr 172.21.15.112:0/377653119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f41c04700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3c1022b0 0x7f8f3c1a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f42405700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3c1082f0 0x7f8f3c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:28.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f41403700 1 -- 172.21.15.112:0/377653119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3c1022b0 msgr2=0x7f8f3c1a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f41403700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f3c1022b0 0x7f8f3c1a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f41403700 1 -- 172.21.15.112:0/377653119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3c1082f0 msgr2=0x7f8f3c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.493+0000 7f8f41403700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3c1082f0 0x7f8f3c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.494+0000 7f8f41403700 1 -- 172.21.15.112:0/377653119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c19b1e0 con 0x7f8f3c102c50 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.494+0000 7f8f42405700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f3c1082f0 0x7f8f3c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.494+0000 7f8f41403700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 0x7f8f3c1a0e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c9b0 tx=0x7f8f2c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.494+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c007b00 con 0x7f8f3c102c50 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.494+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c19b4d0 con 0x7f8f3c102c50 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.494+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c1a7790 con 0x7f8f3c102c50 2024-04-24T23:13:28.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.494+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8f2c007ca0 con 0x7f8f3c102c50 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.494+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c018780 con 0x7f8f3c102c50 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.495+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f3c1014c0 con 0x7f8f3c102c50 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.496+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8f2c00f450 con 0x7f8f3c102c50 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.497+0000 7f8f32ffd700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8f280a2d30 0x7f8f280a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.497+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8f2c0cf8b0 con 0x7f8f3c102c50 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.497+0000 7f8f41c04700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8f280a2d30 0x7f8f280a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.497+0000 7f8f32ffd700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] conn(0x7f8f280ade90 0x7f8f280b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.497+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 --> [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f2c015e70 con 0x7f8f280ade90 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.497+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f8f2c0cfd00 con 0x7f8f3c102c50 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.497+0000 7f8f42405700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] conn(0x7f8f280ade90 0x7f8f280b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.498+0000 7f8f41c04700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8f280a2d30 0x7f8f280a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c19bbc0 tx=0x7f8f3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:28.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.498+0000 7f8f42405700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] conn(0x7f8f280ade90 0x7f8f280b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:28.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.498+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 <== osd.3 v2:172.21.15.112:6810/1421446388 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f2c015e70 con 0x7f8f280ade90 2024-04-24T23:13:28.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.514+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 --> [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f3c1039a0 con 0x7f8f280ade90 2024-04-24T23:13:28.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f32ffd700 1 -- 172.21.15.112:0/377653119 <== osd.3 v2:172.21.15.112:6810/1421446388 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8f3c1039a0 con 0x7f8f280ade90 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] conn(0x7f8f280ade90 msgr2=0x7f8f280b0330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f43e68700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] conn(0x7f8f280ade90 0x7f8f280b0330 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8f280a2d30 msgr2=0x7f8f280a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f43e68700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8f280a2d30 0x7f8f280a51f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c19bbc0 tx=0x7f8f3800b410 comp rx=0 tx=0).stop 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 msgr2=0x7f8f3c1a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f43e68700 1 --2- 172.21.15.112:0/377653119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3c102c50 0x7f8f3c1a0e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c9b0 tx=0x7f8f2c00ccc0 comp rx=0 tx=0).stop 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f42405700 1 -- 172.21.15.112:0/377653119 reap_dead start 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 shutdown_connections 2024-04-24T23:13:28.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.515+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 >> 172.21.15.112:0/377653119 conn(0x7f8f3c079300 msgr2=0x7f8f3c100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:28.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.516+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 shutdown_connections 2024-04-24T23:13:28.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:28.516+0000 7f8f43e68700 1 -- 172.21.15.112:0/377653119 wait complete. 2024-04-24T23:13:28.907 INFO:teuthology.orchestra.run.smithi112.stdout:201863462961 2024-04-24T23:13:28.908 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-24T23:13:29.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:28 smithi112 ceph-mon[29186]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:13:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:28 smithi187 ceph-mon[33292]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:13:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:28 smithi150 ceph-mon[32247]: osdmap e92: 12 total, 12 up, 12 in 2024-04-24T23:13:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 -- 172.21.15.112:0/1484234924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 msgr2=0x7f7d2c06adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 --2- 172.21.15.112:0/1484234924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 0x7f7d2c06adc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7d200099c0 tx=0x7f7d20009cd0 comp rx=0 tx=0).stop 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 -- 172.21.15.112:0/1484234924 shutdown_connections 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 --2- 172.21.15.112:0/1484234924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d2c1082b0 0x7f7d2c108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 --2- 172.21.15.112:0/1484234924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d2c06b300 0x7f7d2c10f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 --2- 172.21.15.112:0/1484234924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 0x7f7d2c06adc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 -- 172.21.15.112:0/1484234924 >> 172.21.15.112:0/1484234924 conn(0x7f7d2c0780d0 msgr2=0x7f7d2c0784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 -- 172.21.15.112:0/1484234924 shutdown_connections 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.504+0000 7f7d324f1700 1 -- 172.21.15.112:0/1484234924 wait complete. 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d324f1700 1 Processor -- start 2024-04-24T23:13:29.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d324f1700 1 -- start start 2024-04-24T23:13:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d324f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 0x7f7d2c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d324f1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d2c06b300 0x7f7d2c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d324f1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d2c1082b0 0x7f7d2c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d324f1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c112600 con 0x7f7d2c06a960 2024-04-24T23:13:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d324f1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c112480 con 0x7f7d2c06b300 2024-04-24T23:13:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d324f1700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c112780 con 0x7f7d2c1082b0 2024-04-24T23:13:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 0x7f7d2c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 0x7f7d2c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56588/0 (socket says 172.21.15.112:56588) 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2bfff700 1 -- 172.21.15.112:0/823916785 learned_addr learned my addr 172.21.15.112:0/823916785 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d30a8e700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d2c1082b0 0x7f7d2c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2b7fe700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d2c06b300 0x7f7d2c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2bfff700 1 -- 172.21.15.112:0/823916785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d2c1082b0 msgr2=0x7f7d2c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2bfff700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d2c1082b0 0x7f7d2c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2bfff700 1 -- 172.21.15.112:0/823916785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d2c06b300 msgr2=0x7f7d2c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2bfff700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d2c06b300 0x7f7d2c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.505+0000 7f7d2bfff700 1 -- 172.21.15.112:0/823916785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c199040 con 0x7f7d2c06a960 2024-04-24T23:13:29.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.506+0000 7f7d2b7fe700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d2c06b300 0x7f7d2c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.506+0000 7f7d30a8e700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d2c1082b0 0x7f7d2c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.506+0000 7f7d2bfff700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 0x7f7d2c19e720 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e9a0 tx=0x7f7d1c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.506+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00cbd0 con 0x7f7d2c06a960 2024-04-24T23:13:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.506+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c199330 con 0x7f7d2c06a960 2024-04-24T23:13:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.506+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d1c00cd70 con 0x7f7d2c06a960 2024-04-24T23:13:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.506+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c018810 con 0x7f7d2c06a960 2024-04-24T23:13:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.506+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c1a55f0 con 0x7f7d2c06a960 2024-04-24T23:13:29.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.507+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d2c0629c0 con 0x7f7d2c06a960 2024-04-24T23:13:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.508+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7d1c0189b0 con 0x7f7d2c06a960 2024-04-24T23:13:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.508+0000 7f7d297fa700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d140a2cb0 0x7f7d140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.509+0000 7f7d2b7fe700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d140a2cb0 0x7f7d140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.509+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7d1c0cffa0 con 0x7f7d2c06a960 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.509+0000 7f7d297fa700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] conn(0x7f7d140ade10 0x7f7d140b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.509+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 --> [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d1c019ec0 con 0x7f7d140ade10 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.509+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c0d03f0 con 0x7f7d2c06a960 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.509+0000 7f7d30a8e700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] conn(0x7f7d140ade10 0x7f7d140b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.509+0000 7f7d2b7fe700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d140a2cb0 0x7f7d140a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006010 tx=0x7f7d20005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.509+0000 7f7d30a8e700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] conn(0x7f7d140ade10 0x7f7d140b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.510+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 <== osd.0 v2:172.21.15.112:6802/3410548369 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d1c019ec0 con 0x7f7d140ade10 2024-04-24T23:13:29.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.526+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 --> [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d2c062dc0 con 0x7f7d140ade10 2024-04-24T23:13:29.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.526+0000 7f7d297fa700 1 -- 172.21.15.112:0/823916785 <== osd.0 v2:172.21.15.112:6802/3410548369 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7d2c062dc0 con 0x7f7d140ade10 2024-04-24T23:13:29.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] conn(0x7f7d140ade10 msgr2=0x7f7d140b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:29.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] conn(0x7f7d140ade10 0x7f7d140b02b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:29.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d140a2cb0 msgr2=0x7f7d140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:29.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d140a2cb0 0x7f7d140a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006010 tx=0x7f7d20005e80 comp rx=0 tx=0).stop 2024-04-24T23:13:29.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 msgr2=0x7f7d2c19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:29.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 --2- 172.21.15.112:0/823916785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d2c06a960 0x7f7d2c19e720 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00e9a0 tx=0x7f7d1c00ed60 comp rx=0 tx=0).stop 2024-04-24T23:13:29.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d30a8e700 1 -- 172.21.15.112:0/823916785 reap_dead start 2024-04-24T23:13:29.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 shutdown_connections 2024-04-24T23:13:29.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 >> 172.21.15.112:0/823916785 conn(0x7f7d2c0780d0 msgr2=0x7f7d2c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:29.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 shutdown_connections 2024-04-24T23:13:29.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:29.527+0000 7f7d324f1700 1 -- 172.21.15.112:0/823916785 wait complete. 2024-04-24T23:13:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:29 smithi187 ceph-mon[33292]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:30.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:29 smithi112 ceph-mon[29186]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:30.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:29 smithi150 ceph-mon[32247]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 383 B/s wr, 9 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:30.331 INFO:teuthology.orchestra.run.smithi112.stdout:77309411398 2024-04-24T23:13:30.331 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T23:13:31.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 -- 172.21.15.112:0/2399806965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c101590 msgr2=0x7f949c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 --2- 172.21.15.112:0/2399806965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c101590 0x7f949c10d7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f949400c210 tx=0x7f949400c520 comp rx=0 tx=0).stop 2024-04-24T23:13:31.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 -- 172.21.15.112:0/2399806965 shutdown_connections 2024-04-24T23:13:31.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 --2- 172.21.15.112:0/2399806965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c101590 0x7f949c10d7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 --2- 172.21.15.112:0/2399806965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949c106980 0x7f949c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 --2- 172.21.15.112:0/2399806965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c100bd0 0x7f949c101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 -- 172.21.15.112:0/2399806965 >> 172.21.15.112:0/2399806965 conn(0x7f949c076450 msgr2=0x7f949c076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:31.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 -- 172.21.15.112:0/2399806965 shutdown_connections 2024-04-24T23:13:31.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.238+0000 7f94a0c42700 1 -- 172.21.15.112:0/2399806965 wait complete. 2024-04-24T23:13:31.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f94a0c42700 1 Processor -- start 2024-04-24T23:13:31.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f94a0c42700 1 -- start start 2024-04-24T23:13:31.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f94a0c42700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949c100bd0 0x7f949c1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f94a0c42700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c101590 0x7f949c196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f94a0c42700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c106980 0x7f949c197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f94a0c42700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f949c1104e0 con 0x7f949c106980 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f94a0c42700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f949c110360 con 0x7f949c101590 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f94a0c42700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f949c110660 con 0x7f949c100bd0 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f949ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c106980 0x7f949c197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f949ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c106980 0x7f949c197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47074/0 (socket says 172.21.15.112:47074) 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.239+0000 7f949ad9d700 1 -- 172.21.15.112:0/1563140165 learned_addr learned my addr 172.21.15.112:0/1563140165 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f9499d9b700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c101590 0x7f949c196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f949a59c700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949c100bd0 0x7f949c1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f949ad9d700 1 -- 172.21.15.112:0/1563140165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949c100bd0 msgr2=0x7f949c1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f949ad9d700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f949c100bd0 0x7f949c1965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f949ad9d700 1 -- 172.21.15.112:0/1563140165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c101590 msgr2=0x7f949c196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f949ad9d700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c101590 0x7f949c196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f949ad9d700 1 -- 172.21.15.112:0/1563140165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949c1a6f10 con 0x7f949c106980 2024-04-24T23:13:31.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f949ad9d700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c106980 0x7f949c197060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f949400dec0 tx=0x7f949400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494021020 con 0x7f949c106980 2024-04-24T23:13:31.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949c1a71a0 con 0x7f949c106980 2024-04-24T23:13:31.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f949400a4e0 con 0x7f949c106980 2024-04-24T23:13:31.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949401c450 con 0x7f949c106980 2024-04-24T23:13:31.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.240+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949c1a7650 con 0x7f949c106980 2024-04-24T23:13:31.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.241+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f949c0ff500 con 0x7f949c106980 2024-04-24T23:13:31.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.242+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9494024020 con 0x7f949c106980 2024-04-24T23:13:31.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.243+0000 7f948b7fe700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f94840a2cb0 0x7f94840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.243+0000 7f949a59c700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f94840a2cb0 0x7f94840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.243+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f94940d0e90 con 0x7f949c106980 2024-04-24T23:13:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.243+0000 7f948b7fe700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] conn(0x7f94840ade10 0x7f94840b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.243+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 --> [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f949400bea0 con 0x7f94840ade10 2024-04-24T23:13:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.243+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f94940d12e0 con 0x7f949c106980 2024-04-24T23:13:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.243+0000 7f9499d9b700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] conn(0x7f94840ade10 0x7f94840b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.243+0000 7f949a59c700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f94840a2cb0 0x7f94840a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f948c005ea0 tx=0x7f948c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.244+0000 7f9499d9b700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] conn(0x7f94840ade10 0x7f94840b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.246 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.245+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 <== osd.9 v2:172.21.15.112:6826/3646247852 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f949400bea0 con 0x7f94840ade10 2024-04-24T23:13:31.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.261+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 --> [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f949c102010 con 0x7f94840ade10 2024-04-24T23:13:31.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.261+0000 7f948b7fe700 1 -- 172.21.15.112:0/1563140165 <== osd.9 v2:172.21.15.112:6826/3646247852 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f949c102010 con 0x7f94840ade10 2024-04-24T23:13:31.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] conn(0x7f94840ade10 msgr2=0x7f94840b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] conn(0x7f94840ade10 0x7f94840b02b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f94840a2cb0 msgr2=0x7f94840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f94840a2cb0 0x7f94840a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f948c005ea0 tx=0x7f948c005e30 comp rx=0 tx=0).stop 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c106980 msgr2=0x7f949c197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 --2- 172.21.15.112:0/1563140165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f949c106980 0x7f949c197060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f949400dec0 tx=0x7f949400def0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f949ad9d700 1 -- 172.21.15.112:0/1563140165 reap_dead start 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 shutdown_connections 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 >> 172.21.15.112:0/1563140165 conn(0x7f949c076450 msgr2=0x7f949c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 shutdown_connections 2024-04-24T23:13:31.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.262+0000 7f94a0c42700 1 -- 172.21.15.112:0/1563140165 wait complete. 2024-04-24T23:13:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 -- 172.21.15.112:0/2056143845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 msgr2=0x7ff8741015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 --2- 172.21.15.112:0/2056143845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 0x7ff8741015e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff868009990 tx=0x7ff868009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 -- 172.21.15.112:0/2056143845 shutdown_connections 2024-04-24T23:13:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 --2- 172.21.15.112:0/2056143845 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff874101b20 0x7ff87410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 --2- 172.21.15.112:0/2056143845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8741071e0 0x7ff8741075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 --2- 172.21.15.112:0/2056143845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 0x7ff8741015e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 -- 172.21.15.112:0/2056143845 >> 172.21.15.112:0/2056143845 conn(0x7ff8740771a0 msgr2=0x7ff8740775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 -- 172.21.15.112:0/2056143845 shutdown_connections 2024-04-24T23:13:31.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.445+0000 7ff879a98700 1 -- 172.21.15.112:0/2056143845 wait complete. 2024-04-24T23:13:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff879a98700 1 Processor -- start 2024-04-24T23:13:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff879a98700 1 -- start start 2024-04-24T23:13:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff879a98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 0x7ff8741999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff879a98700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff874101b20 0x7ff874199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff879a98700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8741071e0 0x7ff8741a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff879a98700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8741115c0 con 0x7ff874101160 2024-04-24T23:13:31.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff879a98700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff874111440 con 0x7ff8741071e0 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff879a98700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff874111740 con 0x7ff874101b20 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.446+0000 7ff8737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 0x7ff8741999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff8737fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 0x7ff8741999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47080/0 (socket says 172.21.15.112:47080) 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff8737fe700 1 -- 172.21.15.112:0/4229144384 learned_addr learned my addr 172.21.15.112:0/4229144384 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff872ffd700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff874101b20 0x7ff874199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff873fff700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8741071e0 0x7ff8741a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff8737fe700 1 -- 172.21.15.112:0/4229144384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff874101b20 msgr2=0x7ff874199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff8737fe700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff874101b20 0x7ff874199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff8737fe700 1 -- 172.21.15.112:0/4229144384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8741071e0 msgr2=0x7ff8741a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff8737fe700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8741071e0 0x7ff8741a4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff8737fe700 1 -- 172.21.15.112:0/4229144384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8741a4f60 con 0x7ff874101160 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff8737fe700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 0x7ff8741999a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff86400e9c0 tx=0x7ff86400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86400cbd0 con 0x7ff874101160 2024-04-24T23:13:31.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8741a5130 con 0x7ff874101160 2024-04-24T23:13:31.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff86400cd70 con 0x7ff874101160 2024-04-24T23:13:31.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864010430 con 0x7ff874101160 2024-04-24T23:13:31.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.447+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8741a5640 con 0x7ff874101160 2024-04-24T23:13:31.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.448+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff874062940 con 0x7ff874101160 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.449+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff864004e50 con 0x7ff874101160 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.450+0000 7ff870ff9700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff85c0a2cb0 0x7ff85c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.450+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff8640cfb80 con 0x7ff874101160 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.450+0000 7ff872ffd700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff85c0a2cb0 0x7ff85c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.450+0000 7ff870ff9700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] conn(0x7ff85c0ade10 0x7ff85c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.450+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 --> [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff86401aec0 con 0x7ff85c0ade10 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.450+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff8640cffd0 con 0x7ff874101160 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.450+0000 7ff872ffd700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff85c0a2cb0 0x7ff85c0a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b2e0 tx=0x7ff868005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.450+0000 7ff873fff700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] conn(0x7ff85c0ade10 0x7ff85c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.451+0000 7ff873fff700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] conn(0x7ff85c0ade10 0x7ff85c0b02b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.452+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 <== osd.10 v2:172.21.15.150:6824/599401021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff86401aec0 con 0x7ff85c0ade10 2024-04-24T23:13:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.468+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 --> [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff874062d70 con 0x7ff85c0ade10 2024-04-24T23:13:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.468+0000 7ff870ff9700 1 -- 172.21.15.112:0/4229144384 <== osd.10 v2:172.21.15.150:6824/599401021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff874062d70 con 0x7ff85c0ade10 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.468+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 >> [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] conn(0x7ff85c0ade10 msgr2=0x7ff85c0b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.468+0000 7ff879a98700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] conn(0x7ff85c0ade10 0x7ff85c0b02b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff85c0a2cb0 msgr2=0x7ff85c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff879a98700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff85c0a2cb0 0x7ff85c0a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff86800b2e0 tx=0x7ff868005960 comp rx=0 tx=0).stop 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 msgr2=0x7ff8741999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff879a98700 1 --2- 172.21.15.112:0/4229144384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff874101160 0x7ff8741999a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff86400e9c0 tx=0x7ff86400ecd0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff873fff700 1 -- 172.21.15.112:0/4229144384 reap_dead start 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 shutdown_connections 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 >> 172.21.15.112:0/4229144384 conn(0x7ff8740771a0 msgr2=0x7ff8740ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 shutdown_connections 2024-04-24T23:13:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.469+0000 7ff879a98700 1 -- 172.21.15.112:0/4229144384 wait complete. 2024-04-24T23:13:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 -- 172.21.15.112:0/1559847537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 msgr2=0x7fb7781086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 --2- 172.21.15.112:0/1559847537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 0x7fb7781086d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb768009990 tx=0x7fb768009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 -- 172.21.15.112:0/1559847537 shutdown_connections 2024-04-24T23:13:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 --2- 172.21.15.112:0/1559847537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb778102c50 0x7fb77810f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 --2- 172.21.15.112:0/1559847537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 0x7fb7781086d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 --2- 172.21.15.112:0/1559847537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7781022b0 0x7fb778102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 -- 172.21.15.112:0/1559847537 >> 172.21.15.112:0/1559847537 conn(0x7fb778079300 msgr2=0x7fb778079710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 -- 172.21.15.112:0/1559847537 shutdown_connections 2024-04-24T23:13:31.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.867+0000 7fb77d1f9700 1 -- 172.21.15.112:0/1559847537 wait complete. 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77d1f9700 1 Processor -- start 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77d1f9700 1 -- start start 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77d1f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7781022b0 0x7fb7781a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77d1f9700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb778102c50 0x7fb7781a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77d1f9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 0x7fb77819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77d1f9700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb778112690 con 0x7fb7781082f0 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77d1f9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb778112510 con 0x7fb7781022b0 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77d1f9700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb778112810 con 0x7fb778102c50 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 0x7fb77819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77759e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 0x7fb77819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47094/0 (socket says 172.21.15.112:47094) 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77759e700 1 -- 172.21.15.112:0/984310881 learned_addr learned my addr 172.21.15.112:0/984310881 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb776d9d700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7781022b0 0x7fb7781a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.868+0000 7fb77659c700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb778102c50 0x7fb7781a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb77759e700 1 -- 172.21.15.112:0/984310881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb778102c50 msgr2=0x7fb7781a0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb77759e700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb778102c50 0x7fb7781a0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb77759e700 1 -- 172.21.15.112:0/984310881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7781022b0 msgr2=0x7fb7781a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb77759e700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7781022b0 0x7fb7781a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb77759e700 1 -- 172.21.15.112:0/984310881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77819b220 con 0x7fb7781082f0 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb776d9d700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb7781022b0 0x7fb7781a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb77759e700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 0x7fb77819aac0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb770007ee0 tx=0x7fb77000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770004bb0 con 0x7fb7781082f0 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77819b510 con 0x7fb7781082f0 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb770004d50 con 0x7fb7781082f0 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7700059b0 con 0x7fb7781082f0 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.869+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7781a77d0 con 0x7fb7781082f0 2024-04-24T23:13:31.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.870+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb77810f120 con 0x7fb7781082f0 2024-04-24T23:13:31.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.871+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb77001a020 con 0x7fb7781082f0 2024-04-24T23:13:31.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.872+0000 7fb75ffff700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb7600a2cb0 0x7fb7600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.872+0000 7fb776d9d700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb7600a2cb0 0x7fb7600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.872+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb7700cf350 con 0x7fb7781082f0 2024-04-24T23:13:31.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.873+0000 7fb75ffff700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] conn(0x7fb7600ade10 0x7fb7600b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:31.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.873+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 --> [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb770017e70 con 0x7fb7600ade10 2024-04-24T23:13:31.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.873+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fb7700cf700 con 0x7fb7781082f0 2024-04-24T23:13:31.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.873+0000 7fb776d9d700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb7600a2cb0 0x7fb7600a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b5c0 tx=0x7fb768005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.873+0000 7fb77659c700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] conn(0x7fb7600ade10 0x7fb7600b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:31.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.873+0000 7fb77659c700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] conn(0x7fb7600ade10 0x7fb7600b02b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:31.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.874+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 <== osd.8 v2:172.21.15.187:6816/3574382245 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb770017e70 con 0x7fb7600ade10 2024-04-24T23:13:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.893+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 --> [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7781039a0 con 0x7fb7600ade10 2024-04-24T23:13:31.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.894+0000 7fb75ffff700 1 -- 172.21.15.112:0/984310881 <== osd.8 v2:172.21.15.187:6816/3574382245 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb7781039a0 con 0x7fb7600ade10 2024-04-24T23:13:31.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.894+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 >> [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] conn(0x7fb7600ade10 msgr2=0x7fb7600b02b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.894+0000 7fb77d1f9700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] conn(0x7fb7600ade10 0x7fb7600b02b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.894+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb7600a2cb0 msgr2=0x7fb7600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.894+0000 7fb77d1f9700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb7600a2cb0 0x7fb7600a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b5c0 tx=0x7fb768005960 comp rx=0 tx=0).stop 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.894+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 msgr2=0x7fb77819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.894+0000 7fb77d1f9700 1 --2- 172.21.15.112:0/984310881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7781082f0 0x7fb77819aac0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb770007ee0 tx=0x7fb77000d3b0 comp rx=0 tx=0).stop 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.895+0000 7fb77759e700 1 -- 172.21.15.112:0/984310881 reap_dead start 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.895+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 shutdown_connections 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.895+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 >> 172.21.15.112:0/984310881 conn(0x7fb778079300 msgr2=0x7fb77810d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.895+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 shutdown_connections 2024-04-24T23:13:31.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:31.895+0000 7fb77d1f9700 1 -- 172.21.15.112:0/984310881 wait complete. 2024-04-24T23:13:32.028 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:31 smithi112 ceph-mon[29186]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 383 B/s wr, 12 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:31 smithi187 ceph-mon[33292]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 383 B/s wr, 12 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:31 smithi150 ceph-mon[32247]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 383 B/s wr, 12 op/s; 22 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:32.434 INFO:teuthology.orchestra.run.smithi112.stdout:223338299435 2024-04-24T23:13:32.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-24T23:13:33.002 INFO:teuthology.orchestra.run.smithi112.stdout:107374182464 2024-04-24T23:13:33.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T23:13:33.420 INFO:teuthology.orchestra.run.smithi112.stdout:292057776164 2024-04-24T23:13:33.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-24T23:13:34.101 INFO:teuthology.orchestra.run.smithi112.stdout:34359738449 2024-04-24T23:13:34.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T23:13:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:33 smithi187 ceph-mon[33292]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T23:13:34.238 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:33 smithi112 ceph-mon[29186]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T23:13:34.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:33 smithi150 ceph-mon[32247]: pgmap v325: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T23:13:34.863 INFO:teuthology.orchestra.run.smithi112.stdout:257698037801 2024-04-24T23:13:34.863 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-24T23:13:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 -- 172.21.15.112:0/3379303755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 msgr2=0x7fa5941086c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 --2- 172.21.15.112:0/3379303755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 0x7fa5941086c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa584009990 tx=0x7fa584009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:35.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 -- 172.21.15.112:0/3379303755 shutdown_connections 2024-04-24T23:13:35.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 --2- 172.21.15.112:0/3379303755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa59406b2d0 0x7fa59410f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:35.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 --2- 172.21.15.112:0/3379303755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 0x7fa5941086c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:35.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 --2- 172.21.15.112:0/3379303755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa59406a930 0x7fa59406ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:35.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 -- 172.21.15.112:0/3379303755 >> 172.21.15.112:0/3379303755 conn(0x7fa594078170 msgr2=0x7fa594078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:35.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 -- 172.21.15.112:0/3379303755 shutdown_connections 2024-04-24T23:13:35.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.030+0000 7fa59a61f700 1 -- 172.21.15.112:0/3379303755 wait complete. 2024-04-24T23:13:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.031+0000 7fa59a61f700 1 Processor -- start 2024-04-24T23:13:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.031+0000 7fa59a61f700 1 -- start start 2024-04-24T23:13:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.031+0000 7fa59a61f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa59406a930 0x7fa594101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.031+0000 7fa59a61f700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa59406b2d0 0x7fa594101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.031+0000 7fa59a61f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 0x7fa594101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:35.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.031+0000 7fa59a61f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa594112660 con 0x7fa5941082e0 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.031+0000 7fa59a61f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941124e0 con 0x7fa59406a930 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.031+0000 7fa59a61f700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941127e0 con 0x7fa59406b2d0 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 0x7fa594101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 0x7fa594101cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47124/0 (socket says 172.21.15.112:47124) 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 -- 172.21.15.112:0/3273617122 learned_addr learned my addr 172.21.15.112:0/3273617122 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa593fff700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa59406a930 0x7fa594101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa5937fe700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa59406b2d0 0x7fa594101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 -- 172.21.15.112:0/3273617122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa59406b2d0 msgr2=0x7fa594101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa59406b2d0 0x7fa594101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:35.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 -- 172.21.15.112:0/3273617122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa59406a930 msgr2=0x7fa594101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:35.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa59406a930 0x7fa594101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:35.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 -- 172.21.15.112:0/3273617122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5941a6ff0 con 0x7fa5941082e0 2024-04-24T23:13:35.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa598bbc700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 0x7fa594101cb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa58c007ee0 tx=0x7fa58c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:35.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00dd20 con 0x7fa5941082e0 2024-04-24T23:13:35.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5941a72e0 con 0x7fa5941082e0 2024-04-24T23:13:35.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa58c004020 con 0x7fa5941082e0 2024-04-24T23:13:35.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c015660 con 0x7fa5941082e0 2024-04-24T23:13:35.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.032+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5941a7820 con 0x7fa5941082e0 2024-04-24T23:13:35.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.034+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa594062940 con 0x7fa5941082e0 2024-04-24T23:13:35.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.035+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa58c0041d0 con 0x7fa5941082e0 2024-04-24T23:13:35.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.036+0000 7fa5917fa700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa57c0ab4c0 0x7fa57c0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:35.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.036+0000 7fa593fff700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa57c0ab4c0 0x7fa57c0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:35.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.036+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fa58c0cf9c0 con 0x7fa5941082e0 2024-04-24T23:13:35.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.036+0000 7fa5917fa700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] conn(0x7fa57c0b6620 0x7fa57c0b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.036+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 --> [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa58c020e50 con 0x7fa57c0b6620 2024-04-24T23:13:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.036+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa58c0cfe10 con 0x7fa5941082e0 2024-04-24T23:13:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.036+0000 7fa593fff700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa57c0ab4c0 0x7fa57c0ad980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b5c0 tx=0x7fa584005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.036+0000 7fa5937fe700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] conn(0x7fa57c0b6620 0x7fa57c0b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.037+0000 7fa5937fe700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] conn(0x7fa57c0b6620 0x7fa57c0b8ac0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:35.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.038+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 <== osd.5 v2:172.21.15.187:6808/1092655909 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa58c020e50 con 0x7fa57c0b6620 2024-04-24T23:13:35.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.057+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 --> [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa594062dc0 con 0x7fa57c0b6620 2024-04-24T23:13:35.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.058+0000 7fa5917fa700 1 -- 172.21.15.112:0/3273617122 <== osd.5 v2:172.21.15.187:6808/1092655909 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa594062dc0 con 0x7fa57c0b6620 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.058+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 >> [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] conn(0x7fa57c0b6620 msgr2=0x7fa57c0b8ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.058+0000 7fa59a61f700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] conn(0x7fa57c0b6620 0x7fa57c0b8ac0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.058+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa57c0ab4c0 msgr2=0x7fa57c0ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.058+0000 7fa59a61f700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa57c0ab4c0 0x7fa57c0ad980 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa58400b5c0 tx=0x7fa584005960 comp rx=0 tx=0).stop 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.058+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 msgr2=0x7fa594101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.058+0000 7fa59a61f700 1 --2- 172.21.15.112:0/3273617122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5941082e0 0x7fa594101cb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa58c007ee0 tx=0x7fa58c00d3b0 comp rx=0 tx=0).stop 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.058+0000 7fa598bbc700 1 -- 172.21.15.112:0/3273617122 reap_dead start 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.059+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 shutdown_connections 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.059+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 >> 172.21.15.112:0/3273617122 conn(0x7fa594078170 msgr2=0x7fa5940ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.059+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 shutdown_connections 2024-04-24T23:13:35.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:35.059+0000 7fa59a61f700 1 -- 172.21.15.112:0/3273617122 wait complete. 2024-04-24T23:13:35.082 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:35.084 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:35.555 INFO:teuthology.orchestra.run.smithi112.stdout:313532612639 2024-04-24T23:13:35.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-24T23:13:35.985 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:35.989 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:36.093 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:35 smithi112 ceph-mon[29186]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T23:13:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:35 smithi187 ceph-mon[33292]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T23:13:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:35 smithi150 ceph-mon[32247]: pgmap v326: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 22 B/s, 1 keys/s, 2 objects/s recovering 2024-04-24T23:13:36.441 INFO:teuthology.orchestra.run.smithi112.stdout:167503724601 2024-04-24T23:13:36.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T23:13:38.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:37 smithi187 ceph-mon[33292]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:38.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:37 smithi112 ceph-mon[29186]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:37 smithi150 ceph-mon[32247]: pgmap v327: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 18 B/s, 1 keys/s, 1 objects/s recovering 2024-04-24T23:13:38.362 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:38.363 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:38.363 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:38.367 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:40.235 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:39 smithi112 ceph-mon[29186]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:13:40.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:39 smithi187 ceph-mon[33292]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:13:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:39 smithi150 ceph-mon[32247]: pgmap v328: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s; 16 B/s, 0 keys/s, 1 objects/s recovering 2024-04-24T23:13:41.268 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:40 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1083006830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:13:41.315 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:40 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1083006830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:13:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:40 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1083006830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-24T23:13:41.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 -- 172.21.15.112:0/4157523022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6441024e0 msgr2=0x7fd6441028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 --2- 172.21.15.112:0/4157523022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6441024e0 0x7fd6441028c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd640009990 tx=0x7fd640009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 -- 172.21.15.112:0/4157523022 shutdown_connections 2024-04-24T23:13:41.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 --2- 172.21.15.112:0/4157523022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd644107570 0x7fd64410f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 --2- 172.21.15.112:0/4157523022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6441024e0 0x7fd6441028c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 --2- 172.21.15.112:0/4157523022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd644102e90 0x7fd644106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 -- 172.21.15.112:0/4157523022 >> 172.21.15.112:0/4157523022 conn(0x7fd644079390 msgr2=0x7fd6440797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 -- 172.21.15.112:0/4157523022 shutdown_connections 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.233+0000 7fd64c560700 1 -- 172.21.15.112:0/4157523022 wait complete. 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64c560700 1 Processor -- start 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64c560700 1 -- start start 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64c560700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6441024e0 0x7fd64419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64c560700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd644102e90 0x7fd64419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64c560700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd644107570 0x7fd644198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64c560700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd644112770 con 0x7fd644107570 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64c560700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441125f0 con 0x7fd644102e90 2024-04-24T23:13:41.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64c560700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6441128f0 con 0x7fd6441024e0 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64aafd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd644107570 0x7fd644198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64aafd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd644107570 0x7fd644198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47146/0 (socket says 172.21.15.112:47146) 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64aafd700 1 -- 172.21.15.112:0/1083006830 learned_addr learned my addr 172.21.15.112:0/1083006830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.234+0000 7fd64a2fc700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6441024e0 0x7fd64419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd649afb700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd644102e90 0x7fd64419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd64aafd700 1 -- 172.21.15.112:0/1083006830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6441024e0 msgr2=0x7fd64419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd64aafd700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6441024e0 0x7fd64419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd64aafd700 1 -- 172.21.15.112:0/1083006830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd644102e90 msgr2=0x7fd64419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd64aafd700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd644102e90 0x7fd64419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd64aafd700 1 -- 172.21.15.112:0/1083006830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6441991b0 con 0x7fd644107570 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd649afb700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd644102e90 0x7fd64419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:13:41.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd64aafd700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd644107570 0x7fd644198a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00b970 tx=0x7fd63c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:41.360 INFO:teuthology.orchestra.run.smithi112.stdout:347892351004 2024-04-24T23:13:41.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd63b7fe700 1 -- 172.21.15.112:0/1083006830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c019070 con 0x7fd644107570 2024-04-24T23:13:41.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd63b7fe700 1 -- 172.21.15.112:0/1083006830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd63c00ce30 con 0x7fd644107570 2024-04-24T23:13:41.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd63b7fe700 1 -- 172.21.15.112:0/1083006830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c0075d0 con 0x7fd644107570 2024-04-24T23:13:41.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6441994a0 con 0x7fd644107570 2024-04-24T23:13:41.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.235+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6441a5760 con 0x7fd644107570 2024-04-24T23:13:41.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.236+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd64410a790 con 0x7fd644107570 2024-04-24T23:13:41.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.238+0000 7fd63b7fe700 1 -- 172.21.15.112:0/1083006830 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd63c01c070 con 0x7fd644107570 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.238+0000 7fd63b7fe700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6300a2cb0 0x7fd6300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.239+0000 7fd64a2fc700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6300a2cb0 0x7fd6300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.239+0000 7fd63b7fe700 1 -- 172.21.15.112:0/1083006830 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd63c0cf210 con 0x7fd644107570 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.239+0000 7fd64a2fc700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6300a2cb0 0x7fd6300a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd64000b5c0 tx=0x7fd640005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.242+0000 7fd63b7fe700 1 -- 172.21.15.112:0/1083006830 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd63c023070 con 0x7fd644107570 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.446+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd644199f80 con 0x7fd644107570 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.446+0000 7fd63b7fe700 1 -- 172.21.15.112:0/1083006830 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd63c09bdc0 con 0x7fd644107570 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6300a2cb0 msgr2=0x7fd6300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6300a2cb0 0x7fd6300a5170 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd64000b5c0 tx=0x7fd640005960 comp rx=0 tx=0).stop 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd644107570 msgr2=0x7fd644198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd644107570 0x7fd644198a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00b970 tx=0x7fd63c00bc80 comp rx=0 tx=0).stop 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 shutdown_connections 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6300a2cb0 0x7fd6300a5170 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd644107570 0x7fd644198a50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6441024e0 0x7fd64419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 --2- 172.21.15.112:0/1083006830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd644102e90 0x7fd64419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 >> 172.21.15.112:0/1083006830 conn(0x7fd644079390 msgr2=0x7fd644100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 shutdown_connections 2024-04-24T23:13:41.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:40.448+0000 7fd64c560700 1 -- 172.21.15.112:0/1083006830 wait complete. 2024-04-24T23:13:41.577 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:42.246 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:41 smithi187 ceph-mon[33292]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:13:42.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:41 smithi112 ceph-mon[29186]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:13:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:41 smithi150 ceph-mon[32247]: pgmap v329: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:13:43.740 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:44.184 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:43 smithi112 ceph-mon[29186]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:13:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:43 smithi150 ceph-mon[32247]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:13:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:43 smithi187 ceph-mon[33292]: pgmap v330: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-24T23:13:44.641 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351001 got 347892351004 for osd.11 2024-04-24T23:13:44.641 DEBUG:teuthology.parallel:result is None 2024-04-24T23:13:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:45 smithi112 ceph-mon[29186]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-24T23:13:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:45 smithi150 ceph-mon[32247]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-24T23:13:46.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:45 smithi187 ceph-mon[33292]: pgmap v331: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-24T23:13:48.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.998+0000 7f0d6f242700 1 -- 172.21.15.112:0/2584316477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 msgr2=0x7f0d68107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.998+0000 7f0d6f242700 1 --2- 172.21.15.112:0/2584316477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 0x7f0d68107580 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009990 tx=0x7f0d5c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 -- 172.21.15.112:0/2584316477 shutdown_connections 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 --2- 172.21.15.112:0/2584316477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68101af0 0x7f0d6810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 --2- 172.21.15.112:0/2584316477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 0x7f0d68107580 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 --2- 172.21.15.112:0/2584316477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d68101130 0x7f0d681015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 -- 172.21.15.112:0/2584316477 >> 172.21.15.112:0/2584316477 conn(0x7f0d680780e0 msgr2=0x7f0d680784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 -- 172.21.15.112:0/2584316477 shutdown_connections 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 -- 172.21.15.112:0/2584316477 wait complete. 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 Processor -- start 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:47.999+0000 7f0d6f242700 1 -- start start 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6f242700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68101130 0x7f0d681998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6f242700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d68101af0 0x7f0d68199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6f242700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 0x7f0d681a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6f242700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68111550 con 0x7f0d681071a0 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6f242700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681113d0 con 0x7f0d68101af0 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6f242700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d681116d0 con 0x7f0d68101130 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 0x7f0d681a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d67fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d68101af0 0x7f0d68199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 0x7f0d681a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54588/0 (socket says 172.21.15.112:54588) 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 -- 172.21.15.112:0/1602683227 learned_addr learned my addr 172.21.15.112:0/1602683227 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:48.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6cfde700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68101130 0x7f0d681998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 -- 172.21.15.112:0/1602683227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68101130 msgr2=0x7f0d681998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68101130 0x7f0d681998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 -- 172.21.15.112:0/1602683227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d68101af0 msgr2=0x7f0d68199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d68101af0 0x7f0d68199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 -- 172.21.15.112:0/1602683227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d681a4e90 con 0x7f0d681071a0 2024-04-24T23:13:48.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d67fff700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d68101af0 0x7f0d68199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.000+0000 7f0d6d7df700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 0x7f0d681a4950 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0d60007ee0 tx=0x7f0d6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.002+0000 7f0d65ffb700 1 -- 172.21.15.112:0/1602683227 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60004bb0 con 0x7f0d681071a0 2024-04-24T23:13:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.002+0000 7f0d65ffb700 1 -- 172.21.15.112:0/1602683227 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0d60004d50 con 0x7f0d681071a0 2024-04-24T23:13:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.002+0000 7f0d65ffb700 1 -- 172.21.15.112:0/1602683227 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d600059b0 con 0x7f0d681071a0 2024-04-24T23:13:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.002+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d68197720 con 0x7f0d681071a0 2024-04-24T23:13:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.002+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d68197c10 con 0x7f0d681071a0 2024-04-24T23:13:48.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.003+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d68068f20 con 0x7f0d681071a0 2024-04-24T23:13:48.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.004+0000 7f0d65ffb700 1 -- 172.21.15.112:0/1602683227 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0d6001a020 con 0x7f0d681071a0 2024-04-24T23:13:48.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.005+0000 7f0d65ffb700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0d580a2d30 0x7f0d580a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.005+0000 7f0d65ffb700 1 -- 172.21.15.112:0/1602683227 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0d600ceb70 con 0x7f0d681071a0 2024-04-24T23:13:48.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.005+0000 7f0d6cfde700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0d580a2d30 0x7f0d580a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.006+0000 7f0d6cfde700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0d580a2d30 0x7f0d580a51f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00b5c0 tx=0x7f0d5c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.011+0000 7f0d65ffb700 1 -- 172.21.15.112:0/1602683227 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0d60010070 con 0x7f0d681071a0 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.216+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0d6802db30 con 0x7f0d681071a0 2024-04-24T23:13:48.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.217+0000 7f0d65ffb700 1 -- 172.21.15.112:0/1602683227 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0d60022250 con 0x7f0d681071a0 2024-04-24T23:13:48.220 INFO:teuthology.orchestra.run.smithi112.stdout:34359738452 2024-04-24T23:13:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0d580a2d30 msgr2=0x7f0d580a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0d580a2d30 0x7f0d580a51f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00b5c0 tx=0x7f0d5c005960 comp rx=0 tx=0).stop 2024-04-24T23:13:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 msgr2=0x7f0d681a4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 0x7f0d681a4950 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0d60007ee0 tx=0x7f0d6000d3b0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 shutdown_connections 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0d580a2d30 0x7f0d580a51f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d681071a0 0x7f0d681a4950 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d68101130 0x7f0d681998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 --2- 172.21.15.112:0/1602683227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d68101af0 0x7f0d68199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 >> 172.21.15.112:0/1602683227 conn(0x7f0d680780e0 msgr2=0x7f0d680ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 shutdown_connections 2024-04-24T23:13:48.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.219+0000 7f0d6f242700 1 -- 172.21.15.112:0/1602683227 wait complete. 2024-04-24T23:13:48.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:48 smithi112 ceph-mon[29186]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:48.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:48 smithi150 ceph-mon[32247]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:48.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:48 smithi187 ceph-mon[33292]: pgmap v332: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:48.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 -- 172.21.15.112:0/3691692686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73941083f0 msgr2=0x7f73941087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 --2- 172.21.15.112:0/3691692686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73941083f0 0x7f73941087d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f739000b0a0 tx=0x7f739000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 -- 172.21.15.112:0/3691692686 shutdown_connections 2024-04-24T23:13:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 --2- 172.21.15.112:0/3691692686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7394102f10 0x7f739410f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 --2- 172.21.15.112:0/3691692686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73941083f0 0x7f73941087d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 --2- 172.21.15.112:0/3691692686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7394102550 0x7f73941029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 -- 172.21.15.112:0/3691692686 >> 172.21.15.112:0/3691692686 conn(0x7f739407a1c0 msgr2=0x7f739407a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 -- 172.21.15.112:0/3691692686 shutdown_connections 2024-04-24T23:13:48.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.567+0000 7f739bf3a700 1 -- 172.21.15.112:0/3691692686 wait complete. 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f739bf3a700 1 Processor -- start 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f739bf3a700 1 -- start start 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f739bf3a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7394102550 0x7f739419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f739bf3a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7394102f10 0x7f739419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f739bf3a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73941083f0 0x7f7394198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f739bf3a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7394112750 con 0x7f7394102550 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f739bf3a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f73941125d0 con 0x7f7394102f10 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f739bf3a700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73941128d0 con 0x7f73941083f0 2024-04-24T23:13:48.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f7399cd6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7394102550 0x7f739419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f7399cd6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7394102550 0x7f739419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54602/0 (socket says 172.21.15.112:54602) 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f7399cd6700 1 -- 172.21.15.112:0/822086299 learned_addr learned my addr 172.21.15.112:0/822086299 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.568+0000 7f73994d5700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7394102f10 0x7f739419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f7399cd6700 1 -- 172.21.15.112:0/822086299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73941083f0 msgr2=0x7f7394198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f7399cd6700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73941083f0 0x7f7394198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f7399cd6700 1 -- 172.21.15.112:0/822086299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7394102f10 msgr2=0x7f739419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f7399cd6700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7394102f10 0x7f739419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f7399cd6700 1 -- 172.21.15.112:0/822086299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7394199070 con 0x7f7394102550 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f7399cd6700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7394102550 0x7f739419e750 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f739000b070 tx=0x7f7390004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f738affd700 1 -- 172.21.15.112:0/822086299 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390021070 con 0x7f7394102550 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f738affd700 1 -- 172.21.15.112:0/822086299 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7390007d40 con 0x7f7394102550 2024-04-24T23:13:48.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7394199300 con 0x7f7394102550 2024-04-24T23:13:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f738affd700 1 -- 172.21.15.112:0/822086299 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73900055a0 con 0x7f7394102550 2024-04-24T23:13:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.569+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941a5620 con 0x7f7394102550 2024-04-24T23:13:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.570+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7394068f20 con 0x7f7394102550 2024-04-24T23:13:48.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.571+0000 7f738affd700 1 -- 172.21.15.112:0/822086299 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7390024070 con 0x7f7394102550 2024-04-24T23:13:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.571+0000 7f738affd700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73800a2cb0 0x7f73800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.572+0000 7f73994d5700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73800a2cb0 0x7f73800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.572+0000 7f738affd700 1 -- 172.21.15.112:0/822086299 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f739000f020 con 0x7f7394102550 2024-04-24T23:13:48.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.572+0000 7f73994d5700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73800a2cb0 0x7f73800a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f739407a750 tx=0x7f7384005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:48.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.575+0000 7f738affd700 1 -- 172.21.15.112:0/822086299 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f739002b020 con 0x7f7394102550 2024-04-24T23:13:48.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.783+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f73941a57f0 con 0x7f7394102550 2024-04-24T23:13:48.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.783+0000 7f738affd700 1 -- 172.21.15.112:0/822086299 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f73900074e0 con 0x7f7394102550 2024-04-24T23:13:48.785 INFO:teuthology.orchestra.run.smithi112.stdout:141733920833 2024-04-24T23:13:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73800a2cb0 msgr2=0x7f73800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73800a2cb0 0x7f73800a5170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f739407a750 tx=0x7f7384005eb0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7394102550 msgr2=0x7f739419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7394102550 0x7f739419e750 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f739000b070 tx=0x7f7390004ed0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 shutdown_connections 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f73800a2cb0 0x7f73800a5170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f73941083f0 0x7f7394198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7394102550 0x7f739419e750 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 --2- 172.21.15.112:0/822086299 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7394102f10 0x7f739419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 >> 172.21.15.112:0/822086299 conn(0x7f739407a1c0 msgr2=0x7f7394101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 shutdown_connections 2024-04-24T23:13:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:48.785+0000 7f739bf3a700 1 -- 172.21.15.112:0/822086299 wait complete. 2024-04-24T23:13:49.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:49 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1602683227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:13:49.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:49 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/822086299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:13:49.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 -- 172.21.15.112:0/1642562346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074460 msgr2=0x7ffad4074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:49.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 --2- 172.21.15.112:0/1642562346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074460 0x7ffad4074840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009990 tx=0x7ffac4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 -- 172.21.15.112:0/1642562346 shutdown_connections 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 --2- 172.21.15.112:0/1642562346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad410ec30 0x7ffad4111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 --2- 172.21.15.112:0/1642562346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074460 0x7ffad4074840 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 --2- 172.21.15.112:0/1642562346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad4074d80 0x7ffad410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 -- 172.21.15.112:0/1642562346 >> 172.21.15.112:0/1642562346 conn(0x7ffad4079390 msgr2=0x7ffad40797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 -- 172.21.15.112:0/1642562346 shutdown_connections 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 -- 172.21.15.112:0/1642562346 wait complete. 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 Processor -- start 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.299+0000 7ffada73a700 1 -- start start 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffada73a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4074460 0x7ffad41a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffada73a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074d80 0x7ffad41a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffada73a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad410ec30 0x7ffad419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffada73a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4114880 con 0x7ffad4074d80 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffada73a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4114700 con 0x7ffad410ec30 2024-04-24T23:13:49.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffada73a700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffad4114a00 con 0x7ffad4074460 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074d80 0x7ffad41a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074d80 0x7ffad41a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54628/0 (socket says 172.21.15.112:54628) 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 -- 172.21.15.112:0/341596406 learned_addr learned my addr 172.21.15.112:0/341596406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad3fff700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4074460 0x7ffad41a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 -- 172.21.15.112:0/341596406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4074460 msgr2=0x7ffad41a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4074460 0x7ffad41a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 -- 172.21.15.112:0/341596406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad410ec30 msgr2=0x7ffad419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad410ec30 0x7ffad419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 -- 172.21.15.112:0/341596406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad419b240 con 0x7ffad4074d80 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad37fe700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074d80 0x7ffad41a0e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffac800d980 tx=0x7ffac800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad17fa700 1 -- 172.21.15.112:0/341596406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8016070 con 0x7ffad4074d80 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad17fa700 1 -- 172.21.15.112:0/341596406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffac8004520 con 0x7ffad4074d80 2024-04-24T23:13:49.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad419b530 con 0x7ffad4074d80 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffad17fa700 1 -- 172.21.15.112:0/341596406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8004e20 con 0x7ffad4074d80 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.300+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41a77f0 con 0x7ffad4074d80 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.301+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad4068f20 con 0x7ffad4074d80 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.303+0000 7ffad17fa700 1 -- 172.21.15.112:0/341596406 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ffac8004020 con 0x7ffad4074d80 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.303+0000 7ffad17fa700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffabc0a2cb0 0x7ffabc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.303+0000 7ffad3fff700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffabc0a2cb0 0x7ffabc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.303+0000 7ffad17fa700 1 -- 172.21.15.112:0/341596406 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ffac80cf460 con 0x7ffad4074d80 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.304+0000 7ffad3fff700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffabc0a2cb0 0x7ffabc0a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffac4006010 tx=0x7ffac400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:49.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.308+0000 7ffad17fa700 1 -- 172.21.15.112:0/341596406 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffac801f020 con 0x7ffad4074d80 2024-04-24T23:13:49.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:49 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1602683227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:13:49.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:49 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/822086299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:13:49.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:49 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1602683227' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:13:49.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:49 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/822086299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:13:49.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.509+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ffad41a7a20 con 0x7ffad4074d80 2024-04-24T23:13:49.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.510+0000 7ffad17fa700 1 -- 172.21.15.112:0/341596406 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffac809bf00 con 0x7ffad4074d80 2024-04-24T23:13:49.512 INFO:teuthology.orchestra.run.smithi112.stdout:77309411403 2024-04-24T23:13:49.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffabc0a2cb0 msgr2=0x7ffabc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:49.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffabc0a2cb0 0x7ffabc0a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffac4006010 tx=0x7ffac400b540 comp rx=0 tx=0).stop 2024-04-24T23:13:49.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074d80 msgr2=0x7ffad41a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:49.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074d80 0x7ffad41a0e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffac800d980 tx=0x7ffac800dd40 comp rx=0 tx=0).stop 2024-04-24T23:13:49.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 shutdown_connections 2024-04-24T23:13:49.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffabc0a2cb0 0x7ffabc0a5170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffad410ec30 0x7ffad419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffad4074460 0x7ffad41a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 --2- 172.21.15.112:0/341596406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffad4074d80 0x7ffad41a0e60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:49.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 >> 172.21.15.112:0/341596406 conn(0x7ffad4079390 msgr2=0x7ffad40ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:49.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 shutdown_connections 2024-04-24T23:13:49.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:49.512+0000 7ffada73a700 1 -- 172.21.15.112:0/341596406 wait complete. 2024-04-24T23:13:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:50 smithi112 ceph-mon[29186]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:13:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/341596406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:13:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:50 smithi150 ceph-mon[32247]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:13:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:50 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/341596406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:13:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:50 smithi187 ceph-mon[33292]: pgmap v333: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:13:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:50 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/341596406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:13:51.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738449 got 34359738452 for osd.0 2024-04-24T23:13:51.905 DEBUG:teuthology.parallel:result is None 2024-04-24T23:13:52.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:52 smithi112 ceph-mon[29186]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:52.305 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:52 smithi187 ceph-mon[33292]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:52 smithi150 ceph-mon[32247]: pgmap v334: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 -- 172.21.15.112:0/3988583061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 msgr2=0x7f284c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 --2- 172.21.15.112:0/3988583061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 0x7f284c1028f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2848009990 tx=0x7f2848009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 -- 172.21.15.112:0/3988583061 shutdown_connections 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 --2- 172.21.15.112:0/3988583061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f284c107590 0x7f284c10f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 --2- 172.21.15.112:0/3988583061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 0x7f284c1028f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 --2- 172.21.15.112:0/3988583061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f284c102ec0 0x7f284c106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 -- 172.21.15.112:0/3988583061 >> 172.21.15.112:0/3988583061 conn(0x7f284c0793a0 msgr2=0x7f284c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 -- 172.21.15.112:0/3988583061 shutdown_connections 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 -- 172.21.15.112:0/3988583061 wait complete. 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 Processor -- start 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.100+0000 7f2853f3a700 1 -- start start 2024-04-24T23:13:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2853f3a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 0x7f284c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2853f3a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f284c102ec0 0x7f284c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2853f3a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f284c107590 0x7f284c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2853f3a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f284c112710 con 0x7f284c102510 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2853f3a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f284c112590 con 0x7f284c107590 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2853f3a700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f284c112890 con 0x7f284c102ec0 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 0x7f284c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 0x7f284c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39850/0 (socket says 172.21.15.112:39850) 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 -- 172.21.15.112:0/1281829392 learned_addr learned my addr 172.21.15.112:0/1281829392 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f28524d7700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f284c107590 0x7f284c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f28514d5700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f284c102ec0 0x7f284c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:52.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 -- 172.21.15.112:0/1281829392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f284c102ec0 msgr2=0x7f284c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:52.527 INFO:teuthology.orchestra.run.smithi112.stdout:55834574928 2024-04-24T23:13:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f284c102ec0 0x7f284c19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 -- 172.21.15.112:0/1281829392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f284c107590 msgr2=0x7f284c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f284c107590 0x7f284c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 -- 172.21.15.112:0/1281829392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c1990b0 con 0x7f284c102510 2024-04-24T23:13:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f28524d7700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f284c107590 0x7f284c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f28514d5700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f284c102ec0 0x7f284c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:52.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2851cd6700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 0x7f284c19e790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2848000c00 tx=0x7f284800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2842ffd700 1 -- 172.21.15.112:0/1281829392 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848027070 con 0x7f284c102510 2024-04-24T23:13:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2842ffd700 1 -- 172.21.15.112:0/1281829392 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f284800fc30 con 0x7f284c102510 2024-04-24T23:13:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c199340 con 0x7f284c102510 2024-04-24T23:13:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2842ffd700 1 -- 172.21.15.112:0/1281829392 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28480177b0 con 0x7f284c102510 2024-04-24T23:13:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.101+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1a5660 con 0x7f284c102510 2024-04-24T23:13:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.102+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c103990 con 0x7f284c102510 2024-04-24T23:13:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.103+0000 7f2842ffd700 1 -- 172.21.15.112:0/1281829392 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f284802a070 con 0x7f284c102510 2024-04-24T23:13:52.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.104+0000 7f2842ffd700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f28380a2cb0 0x7f28380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.104+0000 7f28514d5700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f28380a2cb0 0x7f28380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:52.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.104+0000 7f2842ffd700 1 -- 172.21.15.112:0/1281829392 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f28480d0370 con 0x7f284c102510 2024-04-24T23:13:52.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.104+0000 7f28514d5700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f28380a2cb0 0x7f28380a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f283c005fd0 tx=0x7f283c005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:52.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.108+0000 7f2842ffd700 1 -- 172.21.15.112:0/1281829392 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2848013070 con 0x7f284c102510 2024-04-24T23:13:52.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.313+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f284c02db30 con 0x7f284c102510 2024-04-24T23:13:52.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.313+0000 7f2842ffd700 1 -- 172.21.15.112:0/1281829392 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f284809cea0 con 0x7f284c102510 2024-04-24T23:13:52.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f28380a2cb0 msgr2=0x7f28380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:52.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f28380a2cb0 0x7f28380a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f283c005fd0 tx=0x7f283c005dc0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 msgr2=0x7f284c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:52.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 0x7f284c19e790 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2848000c00 tx=0x7f284800f740 comp rx=0 tx=0).stop 2024-04-24T23:13:52.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 shutdown_connections 2024-04-24T23:13:52.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f28380a2cb0 0x7f28380a5170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f284c107590 0x7f284c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c102510 0x7f284c19e790 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 --2- 172.21.15.112:0/1281829392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f284c102ec0 0x7f284c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:52.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 >> 172.21.15.112:0/1281829392 conn(0x7f284c0793a0 msgr2=0x7f284c1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:52.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 shutdown_connections 2024-04-24T23:13:52.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:52.315+0000 7f2853f3a700 1 -- 172.21.15.112:0/1281829392 wait complete. 2024-04-24T23:13:53.146 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1281829392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:13:53.177 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920833 for osd.4 2024-04-24T23:13:53.178 DEBUG:teuthology.parallel:result is None 2024-04-24T23:13:53.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:53 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1281829392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:13:53.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:53 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1281829392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:13:53.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574928 for osd.1 2024-04-24T23:13:53.863 DEBUG:teuthology.parallel:result is None 2024-04-24T23:13:54.265 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:54 smithi112 ceph-mon[29186]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:13:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:54 smithi150 ceph-mon[32247]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:13:54.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:54 smithi187 ceph-mon[33292]: pgmap v335: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:13:55.250 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:55 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2002226874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:13:55.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:55 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2002226874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:13:55.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 -- 172.21.15.112:0/1952136351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 msgr2=0x7f408c1015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:55.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 --2- 172.21.15.112:0/1952136351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 0x7f408c1015e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f407c009990 tx=0x7f407c009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 -- 172.21.15.112:0/1952136351 shutdown_connections 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 --2- 172.21.15.112:0/1952136351 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f408c101b20 0x7f408c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 --2- 172.21.15.112:0/1952136351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f408c1071d0 0x7f408c1075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 --2- 172.21.15.112:0/1952136351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 0x7f408c1015e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 -- 172.21.15.112:0/1952136351 >> 172.21.15.112:0/1952136351 conn(0x7f408c078170 msgr2=0x7f408c078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 -- 172.21.15.112:0/1952136351 shutdown_connections 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.728+0000 7f40921bb700 1 -- 172.21.15.112:0/1952136351 wait complete. 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f40921bb700 1 Processor -- start 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f40921bb700 1 -- start start 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f40921bb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 0x7f408c19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f40921bb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f408c101b20 0x7f408c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:55.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f40921bb700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f408c1071d0 0x7f408c197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f40921bb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f408c111680 con 0x7f408c101160 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f40921bb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f408c111500 con 0x7f408c101b20 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f40921bb700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f408c111800 con 0x7f408c1071d0 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 0x7f408c19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408b7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 0x7f408c19d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39870/0 (socket says 172.21.15.112:39870) 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408b7fe700 1 -- 172.21.15.112:0/2002226874 learned_addr learned my addr 172.21.15.112:0/2002226874 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408b7fe700 1 -- 172.21.15.112:0/2002226874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f408c1071d0 msgr2=0x7f408c197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408affd700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f408c101b20 0x7f408c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408bfff700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f408c1071d0 0x7f408c197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:55.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408b7fe700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f408c1071d0 0x7f408c197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.462 INFO:teuthology.orchestra.run.smithi112.stdout:107374182469 2024-04-24T23:13:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408b7fe700 1 -- 172.21.15.112:0/2002226874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f408c101b20 msgr2=0x7f408c19dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408b7fe700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f408c101b20 0x7f408c19dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408b7fe700 1 -- 172.21.15.112:0/2002226874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c198080 con 0x7f408c101160 2024-04-24T23:13:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408affd700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f408c101b20 0x7f408c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:55.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.729+0000 7f408bfff700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f408c1071d0 0x7f408c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.730+0000 7f408b7fe700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 0x7f408c19d760 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f408000d9e0 tx=0x7f408000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.730+0000 7f4088ff9700 1 -- 172.21.15.112:0/2002226874 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408001b070 con 0x7f408c101160 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.730+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c198370 con 0x7f408c101160 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.730+0000 7f4088ff9700 1 -- 172.21.15.112:0/2002226874 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f408000ecd0 con 0x7f408c101160 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.730+0000 7f4088ff9700 1 -- 172.21.15.112:0/2002226874 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080014800 con 0x7f408c101160 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.730+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c06a900 con 0x7f408c101160 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.731+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c102880 con 0x7f408c101160 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.732+0000 7f4088ff9700 1 -- 172.21.15.112:0/2002226874 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f408001e070 con 0x7f408c101160 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.732+0000 7f4088ff9700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40740a2cb0 0x7f40740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.732+0000 7f408affd700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40740a2cb0 0x7f40740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.733+0000 7f4088ff9700 1 -- 172.21.15.112:0/2002226874 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f40800d0f20 con 0x7f408c101160 2024-04-24T23:13:55.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.733+0000 7f408affd700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40740a2cb0 0x7f40740a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b5c0 tx=0x7f407c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.736+0000 7f4088ff9700 1 -- 172.21.15.112:0/2002226874 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f408009db60 con 0x7f408c101160 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.948+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f408c068f20 con 0x7f408c101160 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.948+0000 7f4088ff9700 1 -- 172.21.15.112:0/2002226874 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f408009d460 con 0x7f408c101160 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40740a2cb0 msgr2=0x7f40740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40740a2cb0 0x7f40740a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b5c0 tx=0x7f407c005fb0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 msgr2=0x7f408c19d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 0x7f408c19d760 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f408000d9e0 tx=0x7f408000dda0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 shutdown_connections 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40740a2cb0 0x7f40740a5170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f408c1071d0 0x7f408c197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f408c101160 0x7f408c19d760 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 --2- 172.21.15.112:0/2002226874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f408c101b20 0x7f408c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 >> 172.21.15.112:0/2002226874 conn(0x7f408c078170 msgr2=0x7f408c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 shutdown_connections 2024-04-24T23:13:55.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:54.950+0000 7f40921bb700 1 -- 172.21.15.112:0/2002226874 wait complete. 2024-04-24T23:13:55.485 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:55 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2002226874' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:13:55.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 -- 172.21.15.112:0/4286553801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 msgr2=0x7f7888101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 --2- 172.21.15.112:0/4286553801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 0x7f7888101070 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f787400aaa0 tx=0x7f787400adb0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 -- 172.21.15.112:0/4286553801 shutdown_connections 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 --2- 172.21.15.112:0/4286553801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78881069a0 0x7f7888106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 --2- 172.21.15.112:0/4286553801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78881015b0 0x7f788810d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 --2- 172.21.15.112:0/4286553801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 0x7f7888101070 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 -- 172.21.15.112:0/4286553801 >> 172.21.15.112:0/4286553801 conn(0x7f7888076450 msgr2=0x7f7888076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 -- 172.21.15.112:0/4286553801 shutdown_connections 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.880+0000 7f788ef06700 1 -- 172.21.15.112:0/4286553801 wait complete. 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788ef06700 1 Processor -- start 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788ef06700 1 -- start start 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788ef06700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 0x7f788819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:55.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788ef06700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78881015b0 0x7f788819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788ef06700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78881069a0 0x7f78881988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788ef06700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7888110500 con 0x7f7888100bf0 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788ef06700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7888110380 con 0x7f78881069a0 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788ef06700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7888110680 con 0x7f78881015b0 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788cca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 0x7f788819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788cca2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 0x7f788819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39888/0 (socket says 172.21.15.112:39888) 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.881+0000 7f788cca2700 1 -- 172.21.15.112:0/1811800775 learned_addr learned my addr 172.21.15.112:0/1811800775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788d4a3700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78881069a0 0x7f78881988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f7887fff700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78881015b0 0x7f788819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788cca2700 1 -- 172.21.15.112:0/1811800775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78881015b0 msgr2=0x7f788819ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:55.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788cca2700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78881015b0 0x7f788819ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788cca2700 1 -- 172.21.15.112:0/1811800775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78881069a0 msgr2=0x7f78881988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788cca2700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78881069a0 0x7f78881988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788cca2700 1 -- 172.21.15.112:0/1811800775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7888199010 con 0x7f7888100bf0 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788cca2700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 0x7f788819e6a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f787c00d950 tx=0x7f787c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f7885ffb700 1 -- 172.21.15.112:0/1811800775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c016070 con 0x7f7888100bf0 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7888199300 con 0x7f7888100bf0 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f7885ffb700 1 -- 172.21.15.112:0/1811800775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f787c00fd90 con 0x7f7888100bf0 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f7885ffb700 1 -- 172.21.15.112:0/1811800775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c00e610 con 0x7f7888100bf0 2024-04-24T23:13:55.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.882+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78881a5570 con 0x7f7888100bf0 2024-04-24T23:13:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.883+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7888068f20 con 0x7f7888100bf0 2024-04-24T23:13:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.884+0000 7f7885ffb700 1 -- 172.21.15.112:0/1811800775 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f787c019070 con 0x7f7888100bf0 2024-04-24T23:13:55.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.885+0000 7f7885ffb700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f78780a2cb0 0x7f78780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.885+0000 7f7887fff700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f78780a2cb0 0x7f78780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.885+0000 7f7885ffb700 1 -- 172.21.15.112:0/1811800775 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f787c0cefe0 con 0x7f7888100bf0 2024-04-24T23:13:55.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.885+0000 7f7887fff700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f78780a2cb0 0x7f78780a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7888076970 tx=0x7f787400c690 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:55.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:55.889+0000 7f7885ffb700 1 -- 172.21.15.112:0/1811800775 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f787c09ba80 con 0x7f7888100bf0 2024-04-24T23:13:56.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.095+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f788802db30 con 0x7f7888100bf0 2024-04-24T23:13:56.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:56 smithi112 ceph-mon[29186]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:13:56.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.095+0000 7f7885ffb700 1 -- 172.21.15.112:0/1811800775 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f787c09b380 con 0x7f7888100bf0 2024-04-24T23:13:56.098 INFO:teuthology.orchestra.run.smithi112.stdout:167503724605 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f78780a2cb0 msgr2=0x7f78780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f78780a2cb0 0x7f78780a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7888076970 tx=0x7f787400c690 comp rx=0 tx=0).stop 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 msgr2=0x7f788819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 0x7f788819e6a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f787c00d950 tx=0x7f787c00dc60 comp rx=0 tx=0).stop 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 shutdown_connections 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f78780a2cb0 0x7f78780a5170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78881069a0 0x7f78881988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7888100bf0 0x7f788819e6a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 --2- 172.21.15.112:0/1811800775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78881015b0 0x7f788819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 >> 172.21.15.112:0/1811800775 conn(0x7f7888076450 msgr2=0x7f78880feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 shutdown_connections 2024-04-24T23:13:56.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.098+0000 7f788ef06700 1 -- 172.21.15.112:0/1811800775 wait complete. 2024-04-24T23:13:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:56 smithi150 ceph-mon[32247]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:13:56.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:56 smithi187 ceph-mon[33292]: pgmap v336: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:13:56.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 -- 172.21.15.112:0/1061428878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb4102e90 msgr2=0x7feeb4106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:56.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 --2- 172.21.15.112:0/1061428878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb4102e90 0x7feeb4106e90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feea4009990 tx=0x7feea4009ca0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 -- 172.21.15.112:0/1061428878 shutdown_connections 2024-04-24T23:13:56.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 --2- 172.21.15.112:0/1061428878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeb4107570 0x7feeb410f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 --2- 172.21.15.112:0/1061428878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb41024e0 0x7feeb41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 --2- 172.21.15.112:0/1061428878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb4102e90 0x7feeb4106e90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 -- 172.21.15.112:0/1061428878 >> 172.21.15.112:0/1061428878 conn(0x7feeb4079390 msgr2=0x7feeb40797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:56.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 -- 172.21.15.112:0/1061428878 shutdown_connections 2024-04-24T23:13:56.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.533+0000 7feebb65b700 1 -- 172.21.15.112:0/1061428878 wait complete. 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feebb65b700 1 Processor -- start 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feebb65b700 1 -- start start 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feebb65b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb41024e0 0x7feeb4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feebb65b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb4102e90 0x7feeb4199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feebb65b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeb4107570 0x7feeb419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feebb65b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feeb41126b0 con 0x7feeb41024e0 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feebb65b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feeb4112530 con 0x7feeb4107570 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feebb65b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feeb4112830 con 0x7feeb4102e90 2024-04-24T23:13:56.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feeb93f7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb41024e0 0x7feeb4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feeb93f7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb41024e0 0x7feeb4198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39906/0 (socket says 172.21.15.112:39906) 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feeb93f7700 1 -- 172.21.15.112:0/1749491970 learned_addr learned my addr 172.21.15.112:0/1749491970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feeb8bf6700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb4102e90 0x7feeb4199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.534+0000 7feeb9bf8700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeb4107570 0x7feeb419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeb9bf8700 1 -- 172.21.15.112:0/1749491970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb4102e90 msgr2=0x7feeb4199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeb9bf8700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb4102e90 0x7feeb4199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeb9bf8700 1 -- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb41024e0 msgr2=0x7feeb4198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeb9bf8700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb41024e0 0x7feeb4198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeb9bf8700 1 -- 172.21.15.112:0/1749491970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb419db40 con 0x7feeb4107570 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeb93f7700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb41024e0 0x7feeb4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeb9bf8700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeb4107570 0x7feeb419d440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feeac007f10 tx=0x7feeac00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeaa7fc700 1 -- 172.21.15.112:0/1749491970 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac004db0 con 0x7feeb4107570 2024-04-24T23:13:56.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb419de30 con 0x7feeb4107570 2024-04-24T23:13:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeaa7fc700 1 -- 172.21.15.112:0/1749491970 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feeac009e20 con 0x7feeb4107570 2024-04-24T23:13:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feeaa7fc700 1 -- 172.21.15.112:0/1749491970 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac015660 con 0x7feeb4107570 2024-04-24T23:13:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.535+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb41a56f0 con 0x7feeb4107570 2024-04-24T23:13:56.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.536+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb4068f20 con 0x7feeb4107570 2024-04-24T23:13:56.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.538+0000 7feeaa7fc700 1 -- 172.21.15.112:0/1749491970 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7feeac00db20 con 0x7feeb4107570 2024-04-24T23:13:56.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.538+0000 7feeaa7fc700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feea00a2cb0 0x7feea00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:56.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.538+0000 7feeb93f7700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feea00a2cb0 0x7feea00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:56.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.538+0000 7feeaa7fc700 1 -- 172.21.15.112:0/1749491970 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7feeac0cf6c0 con 0x7feeb4107570 2024-04-24T23:13:56.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.539+0000 7feeb93f7700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feea00a2cb0 0x7feea00a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feeb0005950 tx=0x7feeb000b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:56.541 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:13:56.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.542+0000 7feeaa7fc700 1 -- 172.21.15.112:0/1749491970 <== mon.2 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feeac010070 con 0x7feeb4107570 2024-04-24T23:13:56.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.742+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7feeb402db30 con 0x7feeb4107570 2024-04-24T23:13:56.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.743+0000 7feeaa7fc700 1 -- 172.21.15.112:0/1749491970 <== mon.2 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feeac09c100 con 0x7feeb4107570 2024-04-24T23:13:56.745 INFO:teuthology.orchestra.run.smithi112.stdout:257698037805 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.745+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feea00a2cb0 msgr2=0x7feea00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.745+0000 7feebb65b700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feea00a2cb0 0x7feea00a5170 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feeb0005950 tx=0x7feeb000b500 comp rx=0 tx=0).stop 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeb4107570 msgr2=0x7feeb419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeb4107570 0x7feeb419d440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feeac007f10 tx=0x7feeac00d3b0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 shutdown_connections 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feea00a2cb0 0x7feea00a5170 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeb4107570 0x7feeb419d440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feeb41024e0 0x7feeb4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 --2- 172.21.15.112:0/1749491970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeb4102e90 0x7feeb4199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 >> 172.21.15.112:0/1749491970 conn(0x7feeb4079390 msgr2=0x7feeb41086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:56.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 shutdown_connections 2024-04-24T23:13:56.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:56.746+0000 7feebb65b700 1 -- 172.21.15.112:0/1749491970 wait complete. 2024-04-24T23:13:57.153 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411403 for osd.2 2024-04-24T23:13:57.154 DEBUG:teuthology.parallel:result is None 2024-04-24T23:13:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:57 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1811800775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:13:57.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:57 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1749491970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:13:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:57 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1811800775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:13:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:57 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1749491970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:13:57.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:57 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1811800775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:13:57.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:57 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1749491970' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-24T23:13:57.689 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182464 got 107374182469 for osd.3 2024-04-24T23:13:57.690 DEBUG:teuthology.parallel:result is None 2024-04-24T23:13:57.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 -- 172.21.15.112:0/72548745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 msgr2=0x7f959010e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 --2- 172.21.15.112:0/72548745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 0x7f959010e280 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f958800b0d0 tx=0x7f958800b3e0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 -- 172.21.15.112:0/72548745 shutdown_connections 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 --2- 172.21.15.112:0/72548745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9590101af0 0x7f9590101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 --2- 172.21.15.112:0/72548745 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 0x7f959010e280 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 --2- 172.21.15.112:0/72548745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95901024a0 0x7f9590105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 -- 172.21.15.112:0/72548745 >> 172.21.15.112:0/72548745 conn(0x7f9590077730 msgr2=0x7f95900feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 -- 172.21.15.112:0/72548745 shutdown_connections 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.351+0000 7f9594913700 1 -- 172.21.15.112:0/72548745 wait complete. 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f9594913700 1 Processor -- start 2024-04-24T23:13:57.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f9594913700 1 -- start start 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f9594913700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9590101af0 0x7f95900fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f9594913700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95901024a0 0x7f95900fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f9594913700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 0x7f95900fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f9594913700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9590111800 con 0x7f95901066b0 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f9594913700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9590111680 con 0x7f95901024a0 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f9594913700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9590111980 con 0x7f9590101af0 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 0x7f95900fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 0x7f95900fd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39924/0 (socket says 172.21.15.112:39924) 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958ed9d700 1 -- 172.21.15.112:0/3399681201 learned_addr learned my addr 172.21.15.112:0/3399681201 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:13:57.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958e59c700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9590101af0 0x7f95900fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stdout:201863462968 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958dd9b700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95901024a0 0x7f95900fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958ed9d700 1 -- 172.21.15.112:0/3399681201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9590101af0 msgr2=0x7f95900fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958ed9d700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9590101af0 0x7f95900fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958ed9d700 1 -- 172.21.15.112:0/3399681201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95901024a0 msgr2=0x7f95900fd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958ed9d700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95901024a0 0x7f95900fd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.352+0000 7f958ed9d700 1 -- 172.21.15.112:0/3399681201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95901a71d0 con 0x7f95901066b0 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.353+0000 7f958ed9d700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 0x7f95900fd5f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9588007a20 tx=0x7f9588007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.353+0000 7f957f7fe700 1 -- 172.21.15.112:0/3399681201 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958801f070 con 0x7f95901066b0 2024-04-24T23:13:57.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.353+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95901a7460 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.353+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95901a78b0 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.353+0000 7f957f7fe700 1 -- 172.21.15.112:0/3399681201 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9588003c00 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.353+0000 7f957f7fe700 1 -- 172.21.15.112:0/3399681201 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958801b6a0 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.354+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9570005330 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.356+0000 7f957f7fe700 1 -- 172.21.15.112:0/3399681201 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9588022070 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.356+0000 7f957f7fe700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95780a2ca0 0x7f95780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.356+0000 7f958e59c700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95780a2ca0 0x7f95780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.356+0000 7f957f7fe700 1 -- 172.21.15.112:0/3399681201 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f95880d0220 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.357+0000 7f958e59c700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95780a2ca0 0x7f95780a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9580005950 tx=0x7f958000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.360+0000 7f957f7fe700 1 -- 172.21.15.112:0/3399681201 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f958809ceb0 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.562+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f95700058f0 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.562+0000 7f957f7fe700 1 -- 172.21.15.112:0/3399681201 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f958809c7b0 con 0x7f95901066b0 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95780a2ca0 msgr2=0x7f95780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95780a2ca0 0x7f95780a5160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9580005950 tx=0x7f958000b410 comp rx=0 tx=0).stop 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 msgr2=0x7f95900fd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 0x7f95900fd5f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9588007a20 tx=0x7f9588007a50 comp rx=0 tx=0).stop 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 shutdown_connections 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95780a2ca0 0x7f95780a5160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95901066b0 0x7f95900fd5f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9590101af0 0x7f95900fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 --2- 172.21.15.112:0/3399681201 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95901024a0 0x7f95900fd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.564+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 >> 172.21.15.112:0/3399681201 conn(0x7f9590077730 msgr2=0x7f9590100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.565+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 shutdown_connections 2024-04-24T23:13:57.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.565+0000 7f9594913700 1 -- 172.21.15.112:0/3399681201 wait complete. 2024-04-24T23:13:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:58 smithi150 ceph-mon[32247]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:58 smithi150 ceph-mon[32247]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-24T23:13:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:58 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3399681201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:13:58.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:58 smithi187 ceph-mon[33292]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:58.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:58 smithi187 ceph-mon[33292]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-24T23:13:58.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:58 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3399681201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:13:58.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:58 smithi112 ceph-mon[29186]: pgmap v337: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:13:58.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:58 smithi112 ceph-mon[29186]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-24T23:13:58.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:58 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3399681201' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-24T23:13:58.547 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462968 for osd.6 2024-04-24T23:13:58.547 DEBUG:teuthology.parallel:result is None 2024-04-24T23:13:59.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:13:59 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/527413097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:13:59.350 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:13:59 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/527413097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:13:59.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:13:59 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/527413097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-24T23:14:00.100 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037801 got 257698037805 for osd.8 2024-04-24T23:14:00.100 DEBUG:teuthology.parallel:result is None 2024-04-24T23:14:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 -- 172.21.15.112:0/2169914180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406b5b0 msgr2=0x7f24a4106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 --2- 172.21.15.112:0/2169914180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406b5b0 0x7f24a4106ee0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f24980099c0 tx=0x7f2498009cd0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 -- 172.21.15.112:0/2169914180 shutdown_connections 2024-04-24T23:14:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 --2- 172.21.15.112:0/2169914180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a406ac00 0x7f24a406afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 --2- 172.21.15.112:0/2169914180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a4107610 0x7f24a410f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 --2- 172.21.15.112:0/2169914180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406b5b0 0x7f24a4106ee0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 -- 172.21.15.112:0/2169914180 >> 172.21.15.112:0/2169914180 conn(0x7f24a4077180 msgr2=0x7f24a4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 -- 172.21.15.112:0/2169914180 shutdown_connections 2024-04-24T23:14:00.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.300+0000 7f24a89ae700 1 -- 172.21.15.112:0/2169914180 wait complete. 2024-04-24T23:14:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a89ae700 1 Processor -- start 2024-04-24T23:14:00.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a89ae700 1 -- start start 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a89ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406ac00 0x7f24a419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a89ae700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a406b5b0 0x7f24a419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a89ae700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a4107610 0x7f24a4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a89ae700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41126c0 con 0x7f24a406ac00 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a89ae700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4112540 con 0x7f24a406b5b0 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a89ae700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4112840 con 0x7f24a4107610 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406ac00 0x7f24a419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406ac00 0x7f24a419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39968/0 (socket says 172.21.15.112:39968) 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 -- 172.21.15.112:0/1820326036 learned_addr learned my addr 172.21.15.112:0/1820326036 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a2d9d700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a4107610 0x7f24a4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a1d9b700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a406b5b0 0x7f24a419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 -- 172.21.15.112:0/1820326036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a4107610 msgr2=0x7f24a4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a4107610 0x7f24a4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 -- 172.21.15.112:0/1820326036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a406b5b0 msgr2=0x7f24a419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a406b5b0 0x7f24a419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 -- 172.21.15.112:0/1820326036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a4199060 con 0x7f24a406ac00 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.301+0000 7f24a259c700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406ac00 0x7f24a419e740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f249400c920 tx=0x7f249400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.302+0000 7f24937fe700 1 -- 172.21.15.112:0/1820326036 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494007b00 con 0x7f24a406ac00 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.302+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a4199350 con 0x7f24a406ac00 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.302+0000 7f24937fe700 1 -- 172.21.15.112:0/1820326036 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2494004d30 con 0x7f24a406ac00 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.302+0000 7f24937fe700 1 -- 172.21.15.112:0/1820326036 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2494005680 con 0x7f24a406ac00 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.302+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41a5610 con 0x7f24a406ac00 2024-04-24T23:14:00.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.303+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a4068f20 con 0x7f24a406ac00 2024-04-24T23:14:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.305+0000 7f24937fe700 1 -- 172.21.15.112:0/1820326036 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2494004750 con 0x7f24a406ac00 2024-04-24T23:14:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.305+0000 7f24937fe700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f248c0a2c20 0x7f248c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.305+0000 7f24a1d9b700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f248c0a2c20 0x7f248c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:00.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.305+0000 7f24937fe700 1 -- 172.21.15.112:0/1820326036 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24940cf350 con 0x7f24a406ac00 2024-04-24T23:14:00.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.306+0000 7f24a1d9b700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f248c0a2c20 0x7f248c0a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2498009990 tx=0x7f2498005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:00.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.309+0000 7f24937fe700 1 -- 172.21.15.112:0/1820326036 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2494014030 con 0x7f24a406ac00 2024-04-24T23:14:00.311 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:00 smithi112 ceph-mon[29186]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:14:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:00 smithi150 ceph-mon[32247]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:14:00.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 -- 172.21.15.112:0/2531022627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c1075e0 msgr2=0x7f066c10f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 --2- 172.21.15.112:0/2531022627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c1075e0 0x7f066c10f1b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f066400b0d0 tx=0x7f066400b3e0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 -- 172.21.15.112:0/2531022627 shutdown_connections 2024-04-24T23:14:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 --2- 172.21.15.112:0/2531022627 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f066c102510 0x7f066c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 --2- 172.21.15.112:0/2531022627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c1075e0 0x7f066c10f1b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 --2- 172.21.15.112:0/2531022627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f066c102ec0 0x7f066c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 -- 172.21.15.112:0/2531022627 >> 172.21.15.112:0/2531022627 conn(0x7f066c0793e0 msgr2=0x7f066c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:00.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 -- 172.21.15.112:0/2531022627 shutdown_connections 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.879+0000 7f0672509700 1 -- 172.21.15.112:0/2531022627 wait complete. 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f0672509700 1 Processor -- start 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f0672509700 1 -- start start 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f0672509700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c102510 0x7f066c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f0672509700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f066c102ec0 0x7f066c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f0672509700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f066c1075e0 0x7f066c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f0672509700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f066c112780 con 0x7f066c102510 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f0672509700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f066c112600 con 0x7f066c102ec0 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f0672509700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f066c112900 con 0x7f066c1075e0 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f066bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c102510 0x7f066c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f066bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c102510 0x7f066c19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39940/0 (socket says 172.21.15.112:39940) 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f066bfff700 1 -- 172.21.15.112:0/527413097 learned_addr learned my addr 172.21.15.112:0/527413097 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:00.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.880+0000 7f066b7fe700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f066c102ec0 0x7f066c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:00.707 INFO:teuthology.orchestra.run.smithi112.stdout:223338299443 2024-04-24T23:14:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f066bfff700 1 -- 172.21.15.112:0/527413097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f066c1075e0 msgr2=0x7f066c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f066bfff700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f066c1075e0 0x7f066c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f066bfff700 1 -- 172.21.15.112:0/527413097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f066c102ec0 msgr2=0x7f066c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f066bfff700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f066c102ec0 0x7f066c19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f066bfff700 1 -- 172.21.15.112:0/527413097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c199100 con 0x7f066c102510 2024-04-24T23:14:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f066bfff700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c102510 0x7f066c19e790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f065c00b530 tx=0x7f065c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f06697fa700 1 -- 172.21.15.112:0/527413097 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c015020 con 0x7f066c102510 2024-04-24T23:14:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c1993f0 con 0x7f066c102510 2024-04-24T23:14:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f06697fa700 1 -- 172.21.15.112:0/527413097 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f065c011d30 con 0x7f066c102510 2024-04-24T23:14:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f06697fa700 1 -- 172.21.15.112:0/527413097 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c00f2e0 con 0x7f066c102510 2024-04-24T23:14:00.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.881+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c1a5660 con 0x7f066c102510 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.882+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066c068f20 con 0x7f066c102510 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.883+0000 7f06697fa700 1 -- 172.21.15.112:0/527413097 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f065c018020 con 0x7f066c102510 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.884+0000 7f06697fa700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f06540a2c20 0x7f06540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.884+0000 7f066b7fe700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f06540a2c20 0x7f06540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.884+0000 7f06697fa700 1 -- 172.21.15.112:0/527413097 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f065c0cec20 con 0x7f066c102510 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.884+0000 7f066b7fe700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f06540a2c20 0x7f06540a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f066c108a40 tx=0x7f066000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:57.888+0000 7f06697fa700 1 -- 172.21.15.112:0/527413097 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f065c01f020 con 0x7f066c102510 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.091+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f066c02db30 con 0x7f066c102510 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.091+0000 7f06697fa700 1 -- 172.21.15.112:0/527413097 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f065c09b820 con 0x7f066c102510 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.093+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f06540a2c20 msgr2=0x7f06540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.093+0000 7f0672509700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f06540a2c20 0x7f06540a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f066c108a40 tx=0x7f066000b410 comp rx=0 tx=0).stop 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c102510 msgr2=0x7f066c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c102510 0x7f066c19e790 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f065c00b530 tx=0x7f065c00b840 comp rx=0 tx=0).stop 2024-04-24T23:14:00.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 shutdown_connections 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f06540a2c20 0x7f06540a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f066c1075e0 0x7f066c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f066c102510 0x7f066c19e790 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 --2- 172.21.15.112:0/527413097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f066c102ec0 0x7f066c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 >> 172.21.15.112:0/527413097 conn(0x7f066c0793e0 msgr2=0x7f066c100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 shutdown_connections 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:13:58.094+0000 7f0672509700 1 -- 172.21.15.112:0/527413097 wait complete. 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.507+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f24a402db30 con 0x7f24a406ac00 2024-04-24T23:14:00.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.508+0000 7f24937fe700 1 -- 172.21.15.112:0/1820326036 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f249409bf50 con 0x7f24a406ac00 2024-04-24T23:14:00.711 INFO:teuthology.orchestra.run.smithi112.stdout:292057776170 2024-04-24T23:14:00.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f248c0a2c20 msgr2=0x7f248c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f248c0a2c20 0x7f248c0a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2498009990 tx=0x7f2498005fb0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406ac00 msgr2=0x7f24a419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406ac00 0x7f24a419e740 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f249400c920 tx=0x7f249400cc30 comp rx=0 tx=0).stop 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 shutdown_connections 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f248c0a2c20 0x7f248c0a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24a4107610 0x7f24a4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24a406ac00 0x7f24a419e740 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 --2- 172.21.15.112:0/1820326036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f24a406b5b0 0x7f24a419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 >> 172.21.15.112:0/1820326036 conn(0x7f24a4077180 msgr2=0x7f24a4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 shutdown_connections 2024-04-24T23:14:00.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:00.510+0000 7f24a89ae700 1 -- 172.21.15.112:0/1820326036 wait complete. 2024-04-24T23:14:00.713 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:00 smithi187 ceph-mon[33292]: pgmap v338: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-24T23:14:01.056 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299443 for osd.7 2024-04-24T23:14:01.056 DEBUG:teuthology.parallel:result is None 2024-04-24T23:14:01.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:01 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1820326036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:14:01.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:01 smithi150 ceph-mon[32247]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:01.359 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:01 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1820326036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:14:01.360 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:01 smithi112 ceph-mon[29186]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:01.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:01 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1820326036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-24T23:14:01.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:01 smithi187 ceph-mon[33292]: pgmap v339: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:02.117 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724601 got 167503724605 for osd.5 2024-04-24T23:14:02.117 DEBUG:teuthology.parallel:result is None 2024-04-24T23:14:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 -- 172.21.15.112:0/1819540712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 msgr2=0x7f8ce81050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 --2- 172.21.15.112:0/1819540712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 0x7f8ce81050f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000b0a0 tx=0x7f8ce000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 -- 172.21.15.112:0/1819540712 shutdown_connections 2024-04-24T23:14:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 --2- 172.21.15.112:0/1819540712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ce8107150 0x7f8ce806a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 --2- 172.21.15.112:0/1819540712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 0x7f8ce81050f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 --2- 172.21.15.112:0/1819540712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce806ad10 0x7f8ce806b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 -- 172.21.15.112:0/1819540712 >> 172.21.15.112:0/1819540712 conn(0x7f8ce80780a0 msgr2=0x7f8ce80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:02.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 -- 172.21.15.112:0/1819540712 shutdown_connections 2024-04-24T23:14:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.655+0000 7f8cf040a700 1 -- 172.21.15.112:0/1819540712 wait complete. 2024-04-24T23:14:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.656+0000 7f8cf040a700 1 Processor -- start 2024-04-24T23:14:02.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.656+0000 7f8cf040a700 1 -- start start 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.656+0000 7f8cf040a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ce806ad10 0x7f8ce819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.656+0000 7f8cf040a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 0x7f8ce819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.656+0000 7f8cf040a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107150 0x7f8ce81a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.656+0000 7f8cf040a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8118cc0 con 0x7f8ce8104c60 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.656+0000 7f8cf040a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8118b40 con 0x7f8ce806ad10 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.656+0000 7f8cf040a700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8118e40 con 0x7f8ce8107150 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 0x7f8ce819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 0x7f8ce819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36510/0 (socket says 172.21.15.112:36510) 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 -- 172.21.15.112:0/2498681086 learned_addr learned my addr 172.21.15.112:0/2498681086 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:02.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8cee1a6700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ce806ad10 0x7f8ce819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8cee9a7700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107150 0x7f8ce81a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 -- 172.21.15.112:0/2498681086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107150 msgr2=0x7f8ce81a3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107150 0x7f8ce81a3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 -- 172.21.15.112:0/2498681086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ce806ad10 msgr2=0x7f8ce819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ce806ad10 0x7f8ce819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 -- 172.21.15.112:0/2498681086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce81a40a0 con 0x7f8ce8104c60 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8ced9a5700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 0x7f8ce819f5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800b530 tx=0x7f8cd800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8cdf7fe700 1 -- 172.21.15.112:0/2498681086 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8015020 con 0x7f8ce8104c60 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8cdf7fe700 1 -- 172.21.15.112:0/2498681086 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8cd8011d30 con 0x7f8ce8104c60 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce81a4390 con 0x7f8ce8104c60 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8cdf7fe700 1 -- 172.21.15.112:0/2498681086 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800f2e0 con 0x7f8ce8104c60 2024-04-24T23:14:02.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.657+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce81abc10 con 0x7f8ce8104c60 2024-04-24T23:14:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.658+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce8068f20 con 0x7f8ce8104c60 2024-04-24T23:14:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.659+0000 7f8cdf7fe700 1 -- 172.21.15.112:0/2498681086 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8cd8018020 con 0x7f8ce8104c60 2024-04-24T23:14:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.660+0000 7f8cdf7fe700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8cd40a2c20 0x7f8cd40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.660+0000 7f8cee1a6700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8cd40a2c20 0x7f8cd40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:02.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.660+0000 7f8cdf7fe700 1 -- 172.21.15.112:0/2498681086 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8cd80cec40 con 0x7f8ce8104c60 2024-04-24T23:14:02.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.661+0000 7f8cee1a6700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8cd40a2c20 0x7f8cd40a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009c60 tx=0x7f8ce4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:02.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.664+0000 7f8cdf7fe700 1 -- 172.21.15.112:0/2498681086 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8cd801f020 con 0x7f8ce8104c60 2024-04-24T23:14:02.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.884+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8ce81abf00 con 0x7f8ce8104c60 2024-04-24T23:14:02.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.884+0000 7f8cdf7fe700 1 -- 172.21.15.112:0/2498681086 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8cd809b7e0 con 0x7f8ce8104c60 2024-04-24T23:14:02.887 INFO:teuthology.orchestra.run.smithi112.stdout:313532612645 2024-04-24T23:14:02.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.886+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8cd40a2c20 msgr2=0x7f8cd40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.886+0000 7f8cf040a700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8cd40a2c20 0x7f8cd40a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8ce4009c60 tx=0x7f8ce4009260 comp rx=0 tx=0).stop 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.886+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 msgr2=0x7f8ce819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.886+0000 7f8cf040a700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 0x7f8ce819f5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800b530 tx=0x7f8cd800b840 comp rx=0 tx=0).stop 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.887+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 shutdown_connections 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.887+0000 7f8cf040a700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8cd40a2c20 0x7f8cd40a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.887+0000 7f8cf040a700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ce806ad10 0x7f8ce819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.887+0000 7f8cf040a700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107150 0x7f8ce81a3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.887+0000 7f8cf040a700 1 --2- 172.21.15.112:0/2498681086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ce8104c60 0x7f8ce819f5d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.887+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 >> 172.21.15.112:0/2498681086 conn(0x7f8ce80780a0 msgr2=0x7f8ce8103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.887+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 shutdown_connections 2024-04-24T23:14:02.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:02.887+0000 7f8cf040a700 1 -- 172.21.15.112:0/2498681086 wait complete. 2024-04-24T23:14:03.242 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776164 got 292057776170 for osd.9 2024-04-24T23:14:03.242 DEBUG:teuthology.parallel:result is None 2024-04-24T23:14:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:03 smithi187 ceph-mon[33292]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T23:14:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:03 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2498681086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:14:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:03 smithi112 ceph-mon[29186]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T23:14:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:03 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2498681086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:14:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:03 smithi150 ceph-mon[32247]: pgmap v340: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-24T23:14:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:14:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:14:03.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:03 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2498681086' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-24T23:14:03.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612639 got 313532612645 for osd.10 2024-04-24T23:14:03.890 DEBUG:teuthology.parallel:result is None 2024-04-24T23:14:03.891 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T23:14:03.891 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:14:04.136 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:05.941 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:05 smithi112 ceph-mon[29186]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:05.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 -- 172.21.15.112:0/441339250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc1082a0 msgr2=0x7f16bc108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:05.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 --2- 172.21.15.112:0/441339250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc1082a0 0x7f16bc108680 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009990 tx=0x7f16b8009ca0 comp rx=0 tx=0).stop 2024-04-24T23:14:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 -- 172.21.15.112:0/441339250 shutdown_connections 2024-04-24T23:14:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 --2- 172.21.15.112:0/441339250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc06b170 0x7f16bc10f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 --2- 172.21.15.112:0/441339250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc1082a0 0x7f16bc108680 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 --2- 172.21.15.112:0/441339250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16bc06a7d0 0x7f16bc06ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 -- 172.21.15.112:0/441339250 >> 172.21.15.112:0/441339250 conn(0x7f16bc0780a0 msgr2=0x7f16bc0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 -- 172.21.15.112:0/441339250 shutdown_connections 2024-04-24T23:14:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.949+0000 7f16c35c2700 1 -- 172.21.15.112:0/441339250 wait complete. 2024-04-24T23:14:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.950+0000 7f16c35c2700 1 Processor -- start 2024-04-24T23:14:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.950+0000 7f16c35c2700 1 -- start start 2024-04-24T23:14:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.950+0000 7f16c35c2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc06a7d0 0x7f16bc1011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.950+0000 7f16c35c2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc06b170 0x7f16bc101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.950+0000 7f16c35c2700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16bc1082a0 0x7f16bc101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c35c2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc112640 con 0x7f16bc06b170 2024-04-24T23:14:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c35c2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1124c0 con 0x7f16bc06a7d0 2024-04-24T23:14:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c35c2700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc1127c0 con 0x7f16bc1082a0 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc06b170 0x7f16bc101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc06b170 0x7f16bc101730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36530/0 (socket says 172.21.15.112:36530) 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 -- 172.21.15.112:0/1320895304 learned_addr learned my addr 172.21.15.112:0/1320895304 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c135e700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc06a7d0 0x7f16bc1011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c1b5f700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16bc1082a0 0x7f16bc101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 -- 172.21.15.112:0/1320895304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16bc1082a0 msgr2=0x7f16bc101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16bc1082a0 0x7f16bc101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 -- 172.21.15.112:0/1320895304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc06a7d0 msgr2=0x7f16bc1011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc06a7d0 0x7f16bc1011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 -- 172.21.15.112:0/1320895304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1a6fc0 con 0x7f16bc06b170 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c135e700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc06a7d0 0x7f16bc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c0b5d700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc06b170 0x7f16bc101730 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00d980 tx=0x7f16ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16b27fc700 1 -- 172.21.15.112:0/1320895304 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac017070 con 0x7f16bc06b170 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16b27fc700 1 -- 172.21.15.112:0/1320895304 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f16ac004d30 con 0x7f16bc06b170 2024-04-24T23:14:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc1a72b0 con 0x7f16bc06b170 2024-04-24T23:14:05.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16b27fc700 1 -- 172.21.15.112:0/1320895304 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ac005610 con 0x7f16bc06b170 2024-04-24T23:14:05.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.951+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc1a77a0 con 0x7f16bc06b170 2024-04-24T23:14:05.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.953+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc068f20 con 0x7f16bc06b170 2024-04-24T23:14:05.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.953+0000 7f16b27fc700 1 -- 172.21.15.112:0/1320895304 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f16ac010460 con 0x7f16bc06b170 2024-04-24T23:14:05.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.954+0000 7f16b27fc700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f16a80a2c20 0x7f16a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:05.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.954+0000 7f16c135e700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f16a80a2c20 0x7f16a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:05.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.954+0000 7f16b27fc700 1 -- 172.21.15.112:0/1320895304 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f16ac0cf310 con 0x7f16bc06b170 2024-04-24T23:14:05.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.954+0000 7f16c135e700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f16a80a2c20 0x7f16a80a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098e0 tx=0x7f16b800b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:05.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:05.958+0000 7f16b27fc700 1 -- 172.21.15.112:0/1320895304 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f16ac020020 con 0x7f16bc06b170 2024-04-24T23:14:06.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.151+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16bc102720 con 0x7f16a80a2c20 2024-04-24T23:14:06.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.160+0000 7f16b27fc700 1 -- 172.21.15.112:0/1320895304 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340606 (secure 0 0 0) 0x7f16bc102720 con 0x7f16a80a2c20 2024-04-24T23:14:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:06.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f16a80a2c20 msgr2=0x7f16a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:06.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f16a80a2c20 0x7f16a80a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098e0 tx=0x7f16b800b560 comp rx=0 tx=0).stop 2024-04-24T23:14:06.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc06b170 msgr2=0x7f16bc101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:06.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc06b170 0x7f16bc101730 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00d980 tx=0x7f16ac00dd40 comp rx=0 tx=0).stop 2024-04-24T23:14:06.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 shutdown_connections 2024-04-24T23:14:06.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f16a80a2c20 0x7f16a80a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16bc06a7d0 0x7f16bc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16bc1082a0 0x7f16bc101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 --2- 172.21.15.112:0/1320895304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16bc06b170 0x7f16bc101730 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 >> 172.21.15.112:0/1320895304 conn(0x7f16bc0780a0 msgr2=0x7f16bc0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 shutdown_connections 2024-04-24T23:14:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:06.164+0000 7f16c35c2700 1 -- 172.21.15.112:0/1320895304 wait complete. 2024-04-24T23:14:06.171 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-24T23:14:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:05 smithi187 ceph-mon[33292]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:06.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:05 smithi150 ceph-mon[32247]: pgmap v341: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:06.925 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-04-24T23:14:04.809188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661161,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":841,"num_read_kb":548,"num_write":431,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384336,"kb_used_data":14384,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557488,"statfs":{"total":1151940427776,"available":1151546867712,"internally_reserved":0,"allocated":14729216,"data_stored":8023121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":72,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003613"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.819286+0000","last_change":"2024-04-24T23:12:28.013506+0000","last_active":"2024-04-24T23:13:56.819286+0000","last_peered":"2024-04-24T23:13:56.819286+0000","last_clean":"2024-04-24T23:13:56.819286+0000","last_became_active":"2024-04-24T23:12:28.012652+0000","last_became_peered":"2024-04-24T23:12:28.012652+0000","last_unstale":"2024-04-24T23:13:56.819286+0000","last_undegraded":"2024-04-24T23:13:56.819286+0000","last_fullsized":"2024-04-24T23:13:56.819286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:57.420465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816013+0000","last_change":"2024-04-24T23:12:28.004998+0000","last_active":"2024-04-24T23:13:56.816013+0000","last_peered":"2024-04-24T23:13:56.816013+0000","last_clean":"2024-04-24T23:13:56.816013+0000","last_became_active":"2024-04-24T23:12:28.004706+0000","last_became_peered":"2024-04-24T23:12:28.004706+0000","last_unstale":"2024-04-24T23:13:56.816013+0000","last_undegraded":"2024-04-24T23:13:56.816013+0000","last_fullsized":"2024-04-24T23:13:56.816013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.759515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821050+0000","last_change":"2024-04-24T23:12:28.019170+0000","last_active":"2024-04-24T23:13:56.821050+0000","last_peered":"2024-04-24T23:13:56.821050+0000","last_clean":"2024-04-24T23:13:56.821050+0000","last_became_active":"2024-04-24T23:12:28.018815+0000","last_became_peered":"2024-04-24T23:12:28.018815+0000","last_unstale":"2024-04-24T23:13:56.821050+0000","last_undegraded":"2024-04-24T23:13:56.821050+0000","last_fullsized":"2024-04-24T23:13:56.821050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:43.830598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.796500+0000","last_change":"2024-04-24T23:12:28.008662+0000","last_active":"2024-04-24T23:13:56.796500+0000","last_peered":"2024-04-24T23:13:56.796500+0000","last_clean":"2024-04-24T23:13:56.796500+0000","last_became_active":"2024-04-24T23:12:28.008282+0000","last_became_peered":"2024-04-24T23:12:28.008282+0000","last_unstale":"2024-04-24T23:13:56.796500+0000","last_undegraded":"2024-04-24T23:13:56.796500+0000","last_fullsized":"2024-04-24T23:13:56.796500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:17.558899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.802262+0000","last_change":"2024-04-24T23:12:28.005212+0000","last_active":"2024-04-24T23:13:56.802262+0000","last_peered":"2024-04-24T23:13:56.802262+0000","last_clean":"2024-04-24T23:13:56.802262+0000","last_became_active":"2024-04-24T23:12:28.004853+0000","last_became_peered":"2024-04-24T23:12:28.004853+0000","last_unstale":"2024-04-24T23:13:56.802262+0000","last_undegraded":"2024-04-24T23:13:56.802262+0000","last_fullsized":"2024-04-24T23:13:56.802262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:33.650922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.801858+0000","last_change":"2024-04-24T23:12:28.004546+0000","last_active":"2024-04-24T23:13:56.801858+0000","last_peered":"2024-04-24T23:13:56.801858+0000","last_clean":"2024-04-24T23:13:56.801858+0000","last_became_active":"2024-04-24T23:12:28.004194+0000","last_became_peered":"2024-04-24T23:12:28.004194+0000","last_unstale":"2024-04-24T23:13:56.801858+0000","last_undegraded":"2024-04-24T23:13:56.801858+0000","last_fullsized":"2024-04-24T23:13:56.801858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:57.772271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.817292+0000","last_change":"2024-04-24T23:12:28.009436+0000","last_active":"2024-04-24T23:13:56.817292+0000","last_peered":"2024-04-24T23:13:56.817292+0000","last_clean":"2024-04-24T23:13:56.817292+0000","last_became_active":"2024-04-24T23:12:28.008219+0000","last_became_peered":"2024-04-24T23:12:28.008219+0000","last_unstale":"2024-04-24T23:13:56.817292+0000","last_undegraded":"2024-04-24T23:13:56.817292+0000","last_fullsized":"2024-04-24T23:13:56.817292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.808577+0000","last_change":"2024-04-24T23:13:27.957345+0000","last_active":"2024-04-24T23:13:56.808577+0000","last_peered":"2024-04-24T23:13:56.808577+0000","last_clean":"2024-04-24T23:13:56.808577+0000","last_became_active":"2024-04-24T23:13:27.956989+0000","last_became_peered":"2024-04-24T23:13:27.956989+0000","last_unstale":"2024-04-24T23:13:56.808577+0000","last_undegraded":"2024-04-24T23:13:56.808577+0000","last_fullsized":"2024-04-24T23:13:56.808577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:42.120100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.818499+0000","last_change":"2024-04-24T23:12:28.004948+0000","last_active":"2024-04-24T23:13:56.818499+0000","last_peered":"2024-04-24T23:13:56.818499+0000","last_clean":"2024-04-24T23:13:56.818499+0000","last_became_active":"2024-04-24T23:12:28.004380+0000","last_became_peered":"2024-04-24T23:12:28.004380+0000","last_unstale":"2024-04-24T23:13:56.818499+0000","last_undegraded":"2024-04-24T23:13:56.818499+0000","last_fullsized":"2024-04-24T23:13:56.818499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:53.999210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.807670+0000","last_change":"2024-04-24T23:12:28.005192+0000","last_active":"2024-04-24T23:13:56.807670+0000","last_peered":"2024-04-24T23:13:56.807670+0000","last_clean":"2024-04-24T23:13:56.807670+0000","last_became_active":"2024-04-24T23:12:28.004860+0000","last_became_peered":"2024-04-24T23:12:28.004860+0000","last_unstale":"2024-04-24T23:13:56.807670+0000","last_undegraded":"2024-04-24T23:13:56.807670+0000","last_fullsized":"2024-04-24T23:13:56.807670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:21.874408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820929+0000","last_change":"2024-04-24T23:12:28.016098+0000","last_active":"2024-04-24T23:13:56.820929+0000","last_peered":"2024-04-24T23:13:56.820929+0000","last_clean":"2024-04-24T23:13:56.820929+0000","last_became_active":"2024-04-24T23:12:28.015736+0000","last_became_peered":"2024-04-24T23:12:28.015736+0000","last_unstale":"2024-04-24T23:13:56.820929+0000","last_undegraded":"2024-04-24T23:13:56.820929+0000","last_fullsized":"2024-04-24T23:13:56.820929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.071797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.794305+0000","last_change":"2024-04-24T23:12:28.010651+0000","last_active":"2024-04-24T23:13:56.794305+0000","last_peered":"2024-04-24T23:13:56.794305+0000","last_clean":"2024-04-24T23:13:56.794305+0000","last_became_active":"2024-04-24T23:12:28.009700+0000","last_became_peered":"2024-04-24T23:12:28.009700+0000","last_unstale":"2024-04-24T23:13:56.794305+0000","last_undegraded":"2024-04-24T23:13:56.794305+0000","last_fullsized":"2024-04-24T23:13:56.794305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:26.172386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.813453+0000","last_change":"2024-04-24T23:12:28.005743+0000","last_active":"2024-04-24T23:13:56.813453+0000","last_peered":"2024-04-24T23:13:56.813453+0000","last_clean":"2024-04-24T23:13:56.813453+0000","last_became_active":"2024-04-24T23:12:28.005450+0000","last_became_peered":"2024-04-24T23:12:28.005450+0000","last_unstale":"2024-04-24T23:13:56.813453+0000","last_undegraded":"2024-04-24T23:13:56.813453+0000","last_fullsized":"2024-04-24T23:13:56.813453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:14.858212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816039+0000","last_change":"2024-04-24T23:12:28.004140+0000","last_active":"2024-04-24T23:13:56.816039+0000","last_peered":"2024-04-24T23:13:56.816039+0000","last_clean":"2024-04-24T23:13:56.816039+0000","last_became_active":"2024-04-24T23:12:28.003794+0000","last_became_peered":"2024-04-24T23:12:28.003794+0000","last_unstale":"2024-04-24T23:13:56.816039+0000","last_undegraded":"2024-04-24T23:13:56.816039+0000","last_fullsized":"2024-04-24T23:13:56.816039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:02.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.813379+0000","last_change":"2024-04-24T23:12:28.006834+0000","last_active":"2024-04-24T23:13:56.813379+0000","last_peered":"2024-04-24T23:13:56.813379+0000","last_clean":"2024-04-24T23:13:56.813379+0000","last_became_active":"2024-04-24T23:12:28.006504+0000","last_became_peered":"2024-04-24T23:12:28.006504+0000","last_unstale":"2024-04-24T23:13:56.813379+0000","last_undegraded":"2024-04-24T23:13:56.813379+0000","last_fullsized":"2024-04-24T23:13:56.813379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:21.194464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821873+0000","last_change":"2024-04-24T23:12:28.013132+0000","last_active":"2024-04-24T23:13:56.821873+0000","last_peered":"2024-04-24T23:13:56.821873+0000","last_clean":"2024-04-24T23:13:56.821873+0000","last_became_active":"2024-04-24T23:12:28.012751+0000","last_became_peered":"2024-04-24T23:12:28.012751+0000","last_unstale":"2024-04-24T23:13:56.821873+0000","last_undegraded":"2024-04-24T23:13:56.821873+0000","last_fullsized":"2024-04-24T23:13:56.821873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.942985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820913+0000","last_change":"2024-04-24T23:12:28.013719+0000","last_active":"2024-04-24T23:13:56.820913+0000","last_peered":"2024-04-24T23:13:56.820913+0000","last_clean":"2024-04-24T23:13:56.820913+0000","last_became_active":"2024-04-24T23:12:28.013108+0000","last_became_peered":"2024-04-24T23:12:28.013108+0000","last_unstale":"2024-04-24T23:13:56.820913+0000","last_undegraded":"2024-04-24T23:13:56.820913+0000","last_fullsized":"2024-04-24T23:13:56.820913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:38.248867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.819003+0000","last_change":"2024-04-24T23:12:28.012386+0000","last_active":"2024-04-24T23:13:56.819003+0000","last_peered":"2024-04-24T23:13:56.819003+0000","last_clean":"2024-04-24T23:13:56.819003+0000","last_became_active":"2024-04-24T23:12:28.011340+0000","last_became_peered":"2024-04-24T23:12:28.011340+0000","last_unstale":"2024-04-24T23:13:56.819003+0000","last_undegraded":"2024-04-24T23:13:56.819003+0000","last_fullsized":"2024-04-24T23:13:56.819003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:48.115600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.807008+0000","last_change":"2024-04-24T23:12:28.004033+0000","last_active":"2024-04-24T23:13:56.807008+0000","last_peered":"2024-04-24T23:13:56.807008+0000","last_clean":"2024-04-24T23:13:56.807008+0000","last_became_active":"2024-04-24T23:12:28.003648+0000","last_became_peered":"2024-04-24T23:12:28.003648+0000","last_unstale":"2024-04-24T23:13:56.807008+0000","last_undegraded":"2024-04-24T23:13:56.807008+0000","last_fullsized":"2024-04-24T23:13:56.807008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:13.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956921+0000","last_change":"2024-04-24T23:12:32.026590+0000","last_active":"2024-04-24T23:13:27.956921+0000","last_peered":"2024-04-24T23:13:27.956921+0000","last_clean":"2024-04-24T23:13:27.956921+0000","last_became_active":"2024-04-24T23:12:32.026098+0000","last_became_peered":"2024-04-24T23:12:32.026098+0000","last_unstale":"2024-04-24T23:13:27.956921+0000","last_undegraded":"2024-04-24T23:13:27.956921+0000","last_fullsized":"2024-04-24T23:13:27.956921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:55.812641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:00.970420+0000","last_change":"2024-04-24T23:12:27.005325+0000","last_active":"2024-04-24T23:14:00.970420+0000","last_peered":"2024-04-24T23:14:00.970420+0000","last_clean":"2024-04-24T23:14:00.970420+0000","last_became_active":"2024-04-24T23:12:27.004627+0000","last_became_peered":"2024-04-24T23:12:27.004627+0000","last_unstale":"2024-04-24T23:14:00.970420+0000","last_undegraded":"2024-04-24T23:14:00.970420+0000","last_fullsized":"2024-04-24T23:14:00.970420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.985647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.809403+0000","last_change":"2024-04-24T23:12:28.007872+0000","last_active":"2024-04-24T23:13:56.809403+0000","last_peered":"2024-04-24T23:13:56.809403+0000","last_clean":"2024-04-24T23:13:56.809403+0000","last_became_active":"2024-04-24T23:12:28.007493+0000","last_became_peered":"2024-04-24T23:12:28.007493+0000","last_unstale":"2024-04-24T23:13:56.809403+0000","last_undegraded":"2024-04-24T23:13:56.809403+0000","last_fullsized":"2024-04-24T23:13:56.809403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:26.686831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336135+0000","last_change":"2024-04-24T23:12:32.031138+0000","last_active":"2024-04-24T23:13:28.336135+0000","last_peered":"2024-04-24T23:13:28.336135+0000","last_clean":"2024-04-24T23:13:28.336135+0000","last_became_active":"2024-04-24T23:12:32.030626+0000","last_became_peered":"2024-04-24T23:12:32.030626+0000","last_unstale":"2024-04-24T23:13:28.336135+0000","last_undegraded":"2024-04-24T23:13:28.336135+0000","last_fullsized":"2024-04-24T23:13:28.336135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957180+0000","last_change":"2024-04-24T23:12:26.999954+0000","last_active":"2024-04-24T23:13:27.957180+0000","last_peered":"2024-04-24T23:13:27.957180+0000","last_clean":"2024-04-24T23:13:27.957180+0000","last_became_active":"2024-04-24T23:12:26.999507+0000","last_became_peered":"2024-04-24T23:12:26.999507+0000","last_unstale":"2024-04-24T23:13:27.957180+0000","last_undegraded":"2024-04-24T23:13:27.957180+0000","last_fullsized":"2024-04-24T23:13:27.957180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:23.984002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821143+0000","last_change":"2024-04-24T23:12:28.019342+0000","last_active":"2024-04-24T23:13:56.821143+0000","last_peered":"2024-04-24T23:13:56.821143+0000","last_clean":"2024-04-24T23:13:56.821143+0000","last_became_active":"2024-04-24T23:12:28.018976+0000","last_became_peered":"2024-04-24T23:12:28.018976+0000","last_unstale":"2024-04-24T23:13:56.821143+0000","last_undegraded":"2024-04-24T23:13:56.821143+0000","last_fullsized":"2024-04-24T23:13:56.821143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:18.737495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.337245+0000","last_change":"2024-04-24T23:12:32.031015+0000","last_active":"2024-04-24T23:13:28.337245+0000","last_peered":"2024-04-24T23:13:28.337245+0000","last_clean":"2024-04-24T23:13:28.337245+0000","last_became_active":"2024-04-24T23:12:32.030564+0000","last_became_peered":"2024-04-24T23:12:32.030564+0000","last_unstale":"2024-04-24T23:13:28.337245+0000","last_undegraded":"2024-04-24T23:13:28.337245+0000","last_fullsized":"2024-04-24T23:13:28.337245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:12.247397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952539+0000","last_change":"2024-04-24T23:12:27.001229+0000","last_active":"2024-04-24T23:13:27.952539+0000","last_peered":"2024-04-24T23:13:27.952539+0000","last_clean":"2024-04-24T23:13:27.952539+0000","last_became_active":"2024-04-24T23:12:27.000889+0000","last_became_peered":"2024-04-24T23:12:27.000889+0000","last_unstale":"2024-04-24T23:13:27.952539+0000","last_undegraded":"2024-04-24T23:13:27.952539+0000","last_fullsized":"2024-04-24T23:13:27.952539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:39.189828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820422+0000","last_change":"2024-04-24T23:12:28.012309+0000","last_active":"2024-04-24T23:13:56.820422+0000","last_peered":"2024-04-24T23:13:56.820422+0000","last_clean":"2024-04-24T23:13:56.820422+0000","last_became_active":"2024-04-24T23:12:28.011537+0000","last_became_peered":"2024-04-24T23:12:28.011537+0000","last_unstale":"2024-04-24T23:13:56.820422+0000","last_undegraded":"2024-04-24T23:13:56.820422+0000","last_fullsized":"2024-04-24T23:13:56.820422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:38.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336611+0000","last_change":"2024-04-24T23:12:32.030830+0000","last_active":"2024-04-24T23:13:28.336611+0000","last_peered":"2024-04-24T23:13:28.336611+0000","last_clean":"2024-04-24T23:13:28.336611+0000","last_became_active":"2024-04-24T23:12:32.030318+0000","last_became_peered":"2024-04-24T23:12:32.030318+0000","last_unstale":"2024-04-24T23:13:28.336611+0000","last_undegraded":"2024-04-24T23:13:28.336611+0000","last_fullsized":"2024-04-24T23:13:28.336611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:25.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952080+0000","last_change":"2024-04-24T23:12:26.997926+0000","last_active":"2024-04-24T23:13:27.952080+0000","last_peered":"2024-04-24T23:13:27.952080+0000","last_clean":"2024-04-24T23:13:27.952080+0000","last_became_active":"2024-04-24T23:12:26.997552+0000","last_became_peered":"2024-04-24T23:12:26.997552+0000","last_unstale":"2024-04-24T23:13:27.952080+0000","last_undegraded":"2024-04-24T23:13:27.952080+0000","last_fullsized":"2024-04-24T23:13:27.952080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:50.926624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816520+0000","last_change":"2024-04-24T23:13:28.338152+0000","last_active":"2024-04-24T23:13:56.816520+0000","last_peered":"2024-04-24T23:13:56.816520+0000","last_clean":"2024-04-24T23:13:56.816520+0000","last_became_active":"2024-04-24T23:13:28.337728+0000","last_became_peered":"2024-04-24T23:13:28.337728+0000","last_unstale":"2024-04-24T23:13:56.816520+0000","last_undegraded":"2024-04-24T23:13:56.816520+0000","last_fullsized":"2024-04-24T23:13:56.816520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.179108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957588+0000","last_change":"2024-04-24T23:12:32.021601+0000","last_active":"2024-04-24T23:13:27.957588+0000","last_peered":"2024-04-24T23:13:27.957588+0000","last_clean":"2024-04-24T23:13:27.957588+0000","last_became_active":"2024-04-24T23:12:32.021141+0000","last_became_peered":"2024-04-24T23:12:32.021141+0000","last_unstale":"2024-04-24T23:13:27.957588+0000","last_undegraded":"2024-04-24T23:13:27.957588+0000","last_fullsized":"2024-04-24T23:13:27.957588+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:53.795337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:58.969402+0000","last_change":"2024-04-24T23:12:27.002639+0000","last_active":"2024-04-24T23:13:58.969402+0000","last_peered":"2024-04-24T23:13:58.969402+0000","last_clean":"2024-04-24T23:13:58.969402+0000","last_became_active":"2024-04-24T23:12:27.002295+0000","last_became_peered":"2024-04-24T23:12:27.002295+0000","last_unstale":"2024-04-24T23:13:58.969402+0000","last_undegraded":"2024-04-24T23:13:58.969402+0000","last_fullsized":"2024-04-24T23:13:58.969402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:40.962878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.801867+0000","last_change":"2024-04-24T23:12:28.000123+0000","last_active":"2024-04-24T23:13:56.801867+0000","last_peered":"2024-04-24T23:13:56.801867+0000","last_clean":"2024-04-24T23:13:56.801867+0000","last_became_active":"2024-04-24T23:12:27.999778+0000","last_became_peered":"2024-04-24T23:12:27.999778+0000","last_unstale":"2024-04-24T23:13:56.801867+0000","last_undegraded":"2024-04-24T23:13:56.801867+0000","last_fullsized":"2024-04-24T23:13:56.801867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:03.827336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957374+0000","last_change":"2024-04-24T23:12:32.023886+0000","last_active":"2024-04-24T23:13:27.957374+0000","last_peered":"2024-04-24T23:13:27.957374+0000","last_clean":"2024-04-24T23:13:27.957374+0000","last_became_active":"2024-04-24T23:12:32.022796+0000","last_became_peered":"2024-04-24T23:12:32.022796+0000","last_unstale":"2024-04-24T23:13:27.957374+0000","last_undegraded":"2024-04-24T23:13:27.957374+0000","last_fullsized":"2024-04-24T23:13:27.957374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:17.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.951446+0000","last_change":"2024-04-24T23:12:26.999210+0000","last_active":"2024-04-24T23:13:27.951446+0000","last_peered":"2024-04-24T23:13:27.951446+0000","last_clean":"2024-04-24T23:13:27.951446+0000","last_became_active":"2024-04-24T23:12:26.998642+0000","last_became_peered":"2024-04-24T23:12:26.998642+0000","last_unstale":"2024-04-24T23:13:27.951446+0000","last_undegraded":"2024-04-24T23:13:27.951446+0000","last_fullsized":"2024-04-24T23:13:27.951446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:39.171955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.812746+0000","last_change":"2024-04-24T23:13:27.990737+0000","last_active":"2024-04-24T23:13:56.812746+0000","last_peered":"2024-04-24T23:13:56.812746+0000","last_clean":"2024-04-24T23:13:56.812746+0000","last_became_active":"2024-04-24T23:13:27.990428+0000","last_became_peered":"2024-04-24T23:13:27.990428+0000","last_unstale":"2024-04-24T23:13:56.812746+0000","last_undegraded":"2024-04-24T23:13:56.812746+0000","last_fullsized":"2024-04-24T23:13:56.812746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:40.163241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.954317+0000","last_change":"2024-04-24T23:12:32.028192+0000","last_active":"2024-04-24T23:13:27.954317+0000","last_peered":"2024-04-24T23:13:27.954317+0000","last_clean":"2024-04-24T23:13:27.954317+0000","last_became_active":"2024-04-24T23:12:32.027715+0000","last_became_peered":"2024-04-24T23:12:32.027715+0000","last_unstale":"2024-04-24T23:13:27.954317+0000","last_undegraded":"2024-04-24T23:13:27.954317+0000","last_fullsized":"2024-04-24T23:13:27.954317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:22.577980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989453+0000","last_change":"2024-04-24T23:12:27.000212+0000","last_active":"2024-04-24T23:13:27.989453+0000","last_peered":"2024-04-24T23:13:27.989453+0000","last_clean":"2024-04-24T23:13:27.989453+0000","last_became_active":"2024-04-24T23:12:26.999844+0000","last_became_peered":"2024-04-24T23:12:26.999844+0000","last_unstale":"2024-04-24T23:13:27.989453+0000","last_undegraded":"2024-04-24T23:13:27.989453+0000","last_fullsized":"2024-04-24T23:13:27.989453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:00.837670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.804402+0000","last_change":"2024-04-24T23:12:28.002875+0000","last_active":"2024-04-24T23:13:56.804402+0000","last_peered":"2024-04-24T23:13:56.804402+0000","last_clean":"2024-04-24T23:13:56.804402+0000","last_became_active":"2024-04-24T23:12:28.002572+0000","last_became_peered":"2024-04-24T23:12:28.002572+0000","last_unstale":"2024-04-24T23:13:56.804402+0000","last_undegraded":"2024-04-24T23:13:56.804402+0000","last_fullsized":"2024-04-24T23:13:56.804402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:55.931371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955452+0000","last_change":"2024-04-24T23:12:32.024729+0000","last_active":"2024-04-24T23:13:27.955452+0000","last_peered":"2024-04-24T23:13:27.955452+0000","last_clean":"2024-04-24T23:13:27.955452+0000","last_became_active":"2024-04-24T23:12:32.024316+0000","last_became_peered":"2024-04-24T23:12:32.024316+0000","last_unstale":"2024-04-24T23:13:27.955452+0000","last_undegraded":"2024-04-24T23:13:27.955452+0000","last_fullsized":"2024-04-24T23:13:27.955452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:21.668782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955592+0000","last_change":"2024-04-24T23:12:27.003435+0000","last_active":"2024-04-24T23:13:27.955592+0000","last_peered":"2024-04-24T23:13:27.955592+0000","last_clean":"2024-04-24T23:13:27.955592+0000","last_became_active":"2024-04-24T23:12:27.003101+0000","last_became_peered":"2024-04-24T23:12:27.003101+0000","last_unstale":"2024-04-24T23:13:27.955592+0000","last_undegraded":"2024-04-24T23:13:27.955592+0000","last_fullsized":"2024-04-24T23:13:27.955592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:42.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.808426+0000","last_change":"2024-04-24T23:12:28.005056+0000","last_active":"2024-04-24T23:13:56.808426+0000","last_peered":"2024-04-24T23:13:56.808426+0000","last_clean":"2024-04-24T23:13:56.808426+0000","last_became_active":"2024-04-24T23:12:28.004725+0000","last_became_peered":"2024-04-24T23:12:28.004725+0000","last_unstale":"2024-04-24T23:13:56.808426+0000","last_undegraded":"2024-04-24T23:13:56.808426+0000","last_fullsized":"2024-04-24T23:13:56.808426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:49.384950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956393+0000","last_change":"2024-04-24T23:12:32.032947+0000","last_active":"2024-04-24T23:13:27.956393+0000","last_peered":"2024-04-24T23:13:27.956393+0000","last_clean":"2024-04-24T23:13:27.956393+0000","last_became_active":"2024-04-24T23:12:32.032560+0000","last_became_peered":"2024-04-24T23:12:32.032560+0000","last_unstale":"2024-04-24T23:13:27.956393+0000","last_undegraded":"2024-04-24T23:13:27.956393+0000","last_fullsized":"2024-04-24T23:13:27.956393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:29.282938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":170,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:02.974430+0000","last_change":"2024-04-24T23:12:27.000011+0000","last_active":"2024-04-24T23:14:02.974430+0000","last_peered":"2024-04-24T23:14:02.974430+0000","last_clean":"2024-04-24T23:14:02.974430+0000","last_became_active":"2024-04-24T23:12:26.999572+0000","last_became_peered":"2024-04-24T23:12:26.999572+0000","last_unstale":"2024-04-24T23:14:02.974430+0000","last_undegraded":"2024-04-24T23:14:02.974430+0000","last_fullsized":"2024-04-24T23:14:02.974430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:50.453964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820896+0000","last_change":"2024-04-24T23:12:28.012808+0000","last_active":"2024-04-24T23:13:56.820896+0000","last_peered":"2024-04-24T23:13:56.820896+0000","last_clean":"2024-04-24T23:13:56.820896+0000","last_became_active":"2024-04-24T23:12:28.012406+0000","last_became_peered":"2024-04-24T23:12:28.012406+0000","last_unstale":"2024-04-24T23:13:56.820896+0000","last_undegraded":"2024-04-24T23:13:56.820896+0000","last_fullsized":"2024-04-24T23:13:56.820896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:19.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953445+0000","last_change":"2024-04-24T23:12:32.027156+0000","last_active":"2024-04-24T23:13:27.953445+0000","last_peered":"2024-04-24T23:13:27.953445+0000","last_clean":"2024-04-24T23:13:27.953445+0000","last_became_active":"2024-04-24T23:12:32.026743+0000","last_became_peered":"2024-04-24T23:12:32.026743+0000","last_unstale":"2024-04-24T23:13:27.953445+0000","last_undegraded":"2024-04-24T23:13:27.953445+0000","last_fullsized":"2024-04-24T23:13:27.953445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:43.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953121+0000","last_change":"2024-04-24T23:12:27.001832+0000","last_active":"2024-04-24T23:13:27.953121+0000","last_peered":"2024-04-24T23:13:27.953121+0000","last_clean":"2024-04-24T23:13:27.953121+0000","last_became_active":"2024-04-24T23:12:27.001482+0000","last_became_peered":"2024-04-24T23:12:27.001482+0000","last_unstale":"2024-04-24T23:13:27.953121+0000","last_undegraded":"2024-04-24T23:13:27.953121+0000","last_fullsized":"2024-04-24T23:13:27.953121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:28.949767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.813438+0000","last_change":"2024-04-24T23:13:27.960551+0000","last_active":"2024-04-24T23:13:56.813438+0000","last_peered":"2024-04-24T23:13:56.813438+0000","last_clean":"2024-04-24T23:13:56.813438+0000","last_became_active":"2024-04-24T23:13:27.960264+0000","last_became_peered":"2024-04-24T23:13:27.960264+0000","last_unstale":"2024-04-24T23:13:56.813438+0000","last_undegraded":"2024-04-24T23:13:56.813438+0000","last_fullsized":"2024-04-24T23:13:56.813438+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:38.758688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.337104+0000","last_change":"2024-04-24T23:12:32.023007+0000","last_active":"2024-04-24T23:13:28.337104+0000","last_peered":"2024-04-24T23:13:28.337104+0000","last_clean":"2024-04-24T23:13:28.337104+0000","last_became_active":"2024-04-24T23:12:32.021324+0000","last_became_peered":"2024-04-24T23:12:32.021324+0000","last_unstale":"2024-04-24T23:13:28.337104+0000","last_undegraded":"2024-04-24T23:13:28.337104+0000","last_fullsized":"2024-04-24T23:13:28.337104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:41.542778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956673+0000","last_change":"2024-04-24T23:12:26.990562+0000","last_active":"2024-04-24T23:13:27.956673+0000","last_peered":"2024-04-24T23:13:27.956673+0000","last_clean":"2024-04-24T23:13:27.956673+0000","last_became_active":"2024-04-24T23:12:26.990146+0000","last_became_peered":"2024-04-24T23:12:26.990146+0000","last_unstale":"2024-04-24T23:13:27.956673+0000","last_undegraded":"2024-04-24T23:13:27.956673+0000","last_fullsized":"2024-04-24T23:13:27.956673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.621051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.811856+0000","last_change":"2024-04-24T23:12:28.000954+0000","last_active":"2024-04-24T23:13:56.811856+0000","last_peered":"2024-04-24T23:13:56.811856+0000","last_clean":"2024-04-24T23:13:56.811856+0000","last_became_active":"2024-04-24T23:12:28.000610+0000","last_became_peered":"2024-04-24T23:12:28.000610+0000","last_unstale":"2024-04-24T23:13:56.811856+0000","last_undegraded":"2024-04-24T23:13:56.811856+0000","last_fullsized":"2024-04-24T23:13:56.811856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:32.585919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952932+0000","last_change":"2024-04-24T23:12:32.030373+0000","last_active":"2024-04-24T23:13:27.952932+0000","last_peered":"2024-04-24T23:13:27.952932+0000","last_clean":"2024-04-24T23:13:27.952932+0000","last_became_active":"2024-04-24T23:12:32.029954+0000","last_became_peered":"2024-04-24T23:12:32.029954+0000","last_unstale":"2024-04-24T23:13:27.952932+0000","last_undegraded":"2024-04-24T23:13:27.952932+0000","last_fullsized":"2024-04-24T23:13:27.952932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:28.061509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":214,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:02.974345+0000","last_change":"2024-04-24T23:12:26.996719+0000","last_active":"2024-04-24T23:14:02.974345+0000","last_peered":"2024-04-24T23:14:02.974345+0000","last_clean":"2024-04-24T23:14:02.974345+0000","last_became_active":"2024-04-24T23:12:26.996360+0000","last_became_peered":"2024-04-24T23:12:26.996360+0000","last_unstale":"2024-04-24T23:14:02.974345+0000","last_undegraded":"2024-04-24T23:14:02.974345+0000","last_fullsized":"2024-04-24T23:14:02.974345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.526902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":137,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.799339+0000","last_change":"2024-04-24T23:12:28.002889+0000","last_active":"2024-04-24T23:13:56.799339+0000","last_peered":"2024-04-24T23:13:56.799339+0000","last_clean":"2024-04-24T23:13:56.799339+0000","last_became_active":"2024-04-24T23:12:28.002540+0000","last_became_peered":"2024-04-24T23:12:28.002540+0000","last_unstale":"2024-04-24T23:13:56.799339+0000","last_undegraded":"2024-04-24T23:13:56.799339+0000","last_fullsized":"2024-04-24T23:13:56.799339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:57.792628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957499+0000","last_change":"2024-04-24T23:12:32.031160+0000","last_active":"2024-04-24T23:13:27.957499+0000","last_peered":"2024-04-24T23:13:27.957499+0000","last_clean":"2024-04-24T23:13:27.957499+0000","last_became_active":"2024-04-24T23:12:32.030719+0000","last_became_peered":"2024-04-24T23:12:32.030719+0000","last_unstale":"2024-04-24T23:13:27.957499+0000","last_undegraded":"2024-04-24T23:13:27.957499+0000","last_fullsized":"2024-04-24T23:13:27.957499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:45.648011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.967634+0000","last_change":"2024-04-24T23:13:27.967294+0000","last_active":"2024-04-24T23:13:27.967634+0000","last_peered":"2024-04-24T23:13:27.967634+0000","last_clean":"2024-04-24T23:13:27.967634+0000","last_became_active":"2024-04-24T23:13:27.958064+0000","last_became_peered":"2024-04-24T23:13:27.958064+0000","last_unstale":"2024-04-24T23:13:27.967634+0000","last_undegraded":"2024-04-24T23:13:27.967634+0000","last_fullsized":"2024-04-24T23:13:27.967634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:08.824502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816390+0000","last_change":"2024-04-24T23:12:28.003211+0000","last_active":"2024-04-24T23:13:56.816390+0000","last_peered":"2024-04-24T23:13:56.816390+0000","last_clean":"2024-04-24T23:13:56.816390+0000","last_became_active":"2024-04-24T23:12:28.002897+0000","last_became_peered":"2024-04-24T23:12:28.002897+0000","last_unstale":"2024-04-24T23:13:56.816390+0000","last_undegraded":"2024-04-24T23:13:56.816390+0000","last_fullsized":"2024-04-24T23:13:56.816390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.187239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953805+0000","last_change":"2024-04-24T23:12:32.028857+0000","last_active":"2024-04-24T23:13:27.953805+0000","last_peered":"2024-04-24T23:13:27.953805+0000","last_clean":"2024-04-24T23:13:27.953805+0000","last_became_active":"2024-04-24T23:12:32.028455+0000","last_became_peered":"2024-04-24T23:12:32.028455+0000","last_unstale":"2024-04-24T23:13:27.953805+0000","last_undegraded":"2024-04-24T23:13:27.953805+0000","last_fullsized":"2024-04-24T23:13:27.953805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:23.073895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955506+0000","last_change":"2024-04-24T23:12:27.003525+0000","last_active":"2024-04-24T23:13:27.955506+0000","last_peered":"2024-04-24T23:13:27.955506+0000","last_clean":"2024-04-24T23:13:27.955506+0000","last_became_active":"2024-04-24T23:12:27.003203+0000","last_became_peered":"2024-04-24T23:12:27.003203+0000","last_unstale":"2024-04-24T23:13:27.955506+0000","last_undegraded":"2024-04-24T23:13:27.955506+0000","last_fullsized":"2024-04-24T23:13:27.955506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:18.521925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.814230+0000","last_change":"2024-04-24T23:12:28.017111+0000","last_active":"2024-04-24T23:13:56.814230+0000","last_peered":"2024-04-24T23:13:56.814230+0000","last_clean":"2024-04-24T23:13:56.814230+0000","last_became_active":"2024-04-24T23:12:28.016725+0000","last_became_peered":"2024-04-24T23:12:28.016725+0000","last_unstale":"2024-04-24T23:13:56.814230+0000","last_undegraded":"2024-04-24T23:13:56.814230+0000","last_fullsized":"2024-04-24T23:13:56.814230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:25.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953210+0000","last_change":"2024-04-24T23:12:32.027963+0000","last_active":"2024-04-24T23:13:27.953210+0000","last_peered":"2024-04-24T23:13:27.953210+0000","last_clean":"2024-04-24T23:13:27.953210+0000","last_became_active":"2024-04-24T23:12:32.027585+0000","last_became_peered":"2024-04-24T23:12:32.027585+0000","last_unstale":"2024-04-24T23:13:27.953210+0000","last_undegraded":"2024-04-24T23:13:27.953210+0000","last_fullsized":"2024-04-24T23:13:27.953210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:27.111414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":101,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:58.963619+0000","last_change":"2024-04-24T23:12:27.002797+0000","last_active":"2024-04-24T23:13:58.963619+0000","last_peered":"2024-04-24T23:13:58.963619+0000","last_clean":"2024-04-24T23:13:58.963619+0000","last_became_active":"2024-04-24T23:12:27.002086+0000","last_became_peered":"2024-04-24T23:12:27.002086+0000","last_unstale":"2024-04-24T23:13:58.963619+0000","last_undegraded":"2024-04-24T23:13:58.963619+0000","last_fullsized":"2024-04-24T23:13:58.963619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:55.025453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.815909+0000","last_change":"2024-04-24T23:13:27.955130+0000","last_active":"2024-04-24T23:13:56.815909+0000","last_peered":"2024-04-24T23:13:56.815909+0000","last_clean":"2024-04-24T23:13:56.815909+0000","last_became_active":"2024-04-24T23:13:27.954784+0000","last_became_peered":"2024-04-24T23:13:27.954784+0000","last_unstale":"2024-04-24T23:13:56.815909+0000","last_undegraded":"2024-04-24T23:13:56.815909+0000","last_fullsized":"2024-04-24T23:13:56.815909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:10.574771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989872+0000","last_change":"2024-04-24T23:12:32.029104+0000","last_active":"2024-04-24T23:13:27.989872+0000","last_peered":"2024-04-24T23:13:27.989872+0000","last_clean":"2024-04-24T23:13:27.989872+0000","last_became_active":"2024-04-24T23:12:32.028704+0000","last_became_peered":"2024-04-24T23:12:32.028704+0000","last_unstale":"2024-04-24T23:13:27.989872+0000","last_undegraded":"2024-04-24T23:13:27.989872+0000","last_fullsized":"2024-04-24T23:13:27.989872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:33.200884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.977205+0000","last_change":"2024-04-24T23:13:27.976835+0000","last_active":"2024-04-24T23:13:27.977205+0000","last_peered":"2024-04-24T23:13:27.977205+0000","last_clean":"2024-04-24T23:13:27.977205+0000","last_became_active":"2024-04-24T23:13:27.959684+0000","last_became_peered":"2024-04-24T23:13:27.959684+0000","last_unstale":"2024-04-24T23:13:27.977205+0000","last_undegraded":"2024-04-24T23:13:27.977205+0000","last_fullsized":"2024-04-24T23:13:27.977205+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:24.327964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.811557+0000","last_change":"2024-04-24T23:12:28.010510+0000","last_active":"2024-04-24T23:13:56.811557+0000","last_peered":"2024-04-24T23:13:56.811557+0000","last_clean":"2024-04-24T23:13:56.811557+0000","last_became_active":"2024-04-24T23:12:28.009244+0000","last_became_peered":"2024-04-24T23:12:28.009244+0000","last_unstale":"2024-04-24T23:13:56.811557+0000","last_undegraded":"2024-04-24T23:13:56.811557+0000","last_fullsized":"2024-04-24T23:13:56.811557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.146563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955268+0000","last_change":"2024-04-24T23:12:32.031156+0000","last_active":"2024-04-24T23:13:27.955268+0000","last_peered":"2024-04-24T23:13:27.955268+0000","last_clean":"2024-04-24T23:13:27.955268+0000","last_became_active":"2024-04-24T23:12:32.030653+0000","last_became_peered":"2024-04-24T23:12:32.030653+0000","last_unstale":"2024-04-24T23:13:27.955268+0000","last_undegraded":"2024-04-24T23:13:27.955268+0000","last_fullsized":"2024-04-24T23:13:27.955268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:48.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952365+0000","last_change":"2024-04-24T23:12:26.998003+0000","last_active":"2024-04-24T23:13:27.952365+0000","last_peered":"2024-04-24T23:13:27.952365+0000","last_clean":"2024-04-24T23:13:27.952365+0000","last_became_active":"2024-04-24T23:12:26.997661+0000","last_became_peered":"2024-04-24T23:12:26.997661+0000","last_unstale":"2024-04-24T23:13:27.952365+0000","last_undegraded":"2024-04-24T23:13:27.952365+0000","last_fullsized":"2024-04-24T23:13:27.952365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:17.995397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.806718+0000","last_change":"2024-04-24T23:12:28.004822+0000","last_active":"2024-04-24T23:13:56.806718+0000","last_peered":"2024-04-24T23:13:56.806718+0000","last_clean":"2024-04-24T23:13:56.806718+0000","last_became_active":"2024-04-24T23:12:28.004334+0000","last_became_peered":"2024-04-24T23:12:28.004334+0000","last_unstale":"2024-04-24T23:13:56.806718+0000","last_undegraded":"2024-04-24T23:13:56.806718+0000","last_fullsized":"2024-04-24T23:13:56.806718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.717796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952075+0000","last_change":"2024-04-24T23:12:32.085309+0000","last_active":"2024-04-24T23:13:27.952075+0000","last_peered":"2024-04-24T23:13:27.952075+0000","last_clean":"2024-04-24T23:13:27.952075+0000","last_became_active":"2024-04-24T23:12:32.084892+0000","last_became_peered":"2024-04-24T23:12:32.084892+0000","last_unstale":"2024-04-24T23:13:27.952075+0000","last_undegraded":"2024-04-24T23:13:27.952075+0000","last_fullsized":"2024-04-24T23:13:27.952075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:20.318901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953203+0000","last_change":"2024-04-24T23:12:27.005096+0000","last_active":"2024-04-24T23:13:27.953203+0000","last_peered":"2024-04-24T23:13:27.953203+0000","last_clean":"2024-04-24T23:13:27.953203+0000","last_became_active":"2024-04-24T23:12:27.004699+0000","last_became_peered":"2024-04-24T23:12:27.004699+0000","last_unstale":"2024-04-24T23:13:27.953203+0000","last_undegraded":"2024-04-24T23:13:27.953203+0000","last_fullsized":"2024-04-24T23:13:27.953203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:58.225361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.808651+0000","last_change":"2024-04-24T23:12:28.012230+0000","last_active":"2024-04-24T23:13:56.808651+0000","last_peered":"2024-04-24T23:13:56.808651+0000","last_clean":"2024-04-24T23:13:56.808651+0000","last_became_active":"2024-04-24T23:12:28.011900+0000","last_became_peered":"2024-04-24T23:12:28.011900+0000","last_unstale":"2024-04-24T23:13:56.808651+0000","last_undegraded":"2024-04-24T23:13:56.808651+0000","last_fullsized":"2024-04-24T23:13:56.808651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.780548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989677+0000","last_change":"2024-04-24T23:12:32.026388+0000","last_active":"2024-04-24T23:13:27.989677+0000","last_peered":"2024-04-24T23:13:27.989677+0000","last_clean":"2024-04-24T23:13:27.989677+0000","last_became_active":"2024-04-24T23:12:32.025994+0000","last_became_peered":"2024-04-24T23:12:32.025994+0000","last_unstale":"2024-04-24T23:13:27.989677+0000","last_undegraded":"2024-04-24T23:13:27.989677+0000","last_fullsized":"2024-04-24T23:13:27.989677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:13.062781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":98,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:58.967786+0000","last_change":"2024-04-24T23:12:27.005080+0000","last_active":"2024-04-24T23:13:58.967786+0000","last_peered":"2024-04-24T23:13:58.967786+0000","last_clean":"2024-04-24T23:13:58.967786+0000","last_became_active":"2024-04-24T23:12:27.004740+0000","last_became_peered":"2024-04-24T23:12:27.004740+0000","last_unstale":"2024-04-24T23:13:58.967786+0000","last_undegraded":"2024-04-24T23:13:58.967786+0000","last_fullsized":"2024-04-24T23:13:58.967786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:22.502399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.791819+0000","last_change":"2024-04-24T23:12:28.008016+0000","last_active":"2024-04-24T23:13:56.791819+0000","last_peered":"2024-04-24T23:13:56.791819+0000","last_clean":"2024-04-24T23:13:56.791819+0000","last_became_active":"2024-04-24T23:12:28.007139+0000","last_became_peered":"2024-04-24T23:12:28.007139+0000","last_unstale":"2024-04-24T23:13:56.791819+0000","last_undegraded":"2024-04-24T23:13:56.791819+0000","last_fullsized":"2024-04-24T23:13:56.791819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.698193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989740+0000","last_change":"2024-04-24T23:12:32.085290+0000","last_active":"2024-04-24T23:13:27.989740+0000","last_peered":"2024-04-24T23:13:27.989740+0000","last_clean":"2024-04-24T23:13:27.989740+0000","last_became_active":"2024-04-24T23:12:32.084852+0000","last_became_peered":"2024-04-24T23:12:32.084852+0000","last_unstale":"2024-04-24T23:13:27.989740+0000","last_undegraded":"2024-04-24T23:13:27.989740+0000","last_fullsized":"2024-04-24T23:13:27.989740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.753633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.811720+0000","last_change":"2024-04-24T23:12:28.007552+0000","last_active":"2024-04-24T23:13:56.811720+0000","last_peered":"2024-04-24T23:13:56.811720+0000","last_clean":"2024-04-24T23:13:56.811720+0000","last_became_active":"2024-04-24T23:12:28.007196+0000","last_became_peered":"2024-04-24T23:12:28.007196+0000","last_unstale":"2024-04-24T23:13:56.811720+0000","last_undegraded":"2024-04-24T23:13:56.811720+0000","last_fullsized":"2024-04-24T23:13:56.811720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:15.695485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":99,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:57.961172+0000","last_change":"2024-04-24T23:12:26.990814+0000","last_active":"2024-04-24T23:13:57.961172+0000","last_peered":"2024-04-24T23:13:57.961172+0000","last_clean":"2024-04-24T23:13:57.961172+0000","last_became_active":"2024-04-24T23:12:26.990476+0000","last_became_peered":"2024-04-24T23:12:26.990476+0000","last_unstale":"2024-04-24T23:13:57.961172+0000","last_undegraded":"2024-04-24T23:13:57.961172+0000","last_fullsized":"2024-04-24T23:13:57.961172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:32.645770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953123+0000","last_change":"2024-04-24T23:12:32.022039+0000","last_active":"2024-04-24T23:13:27.953123+0000","last_peered":"2024-04-24T23:13:27.953123+0000","last_clean":"2024-04-24T23:13:27.953123+0000","last_became_active":"2024-04-24T23:12:32.021514+0000","last_became_peered":"2024-04-24T23:12:32.021514+0000","last_unstale":"2024-04-24T23:13:27.953123+0000","last_undegraded":"2024-04-24T23:13:27.953123+0000","last_fullsized":"2024-04-24T23:13:27.953123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:42.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.804050+0000","last_change":"2024-04-24T23:12:28.008676+0000","last_active":"2024-04-24T23:13:56.804050+0000","last_peered":"2024-04-24T23:13:56.804050+0000","last_clean":"2024-04-24T23:13:56.804050+0000","last_became_active":"2024-04-24T23:12:28.008376+0000","last_became_peered":"2024-04-24T23:12:28.008376+0000","last_unstale":"2024-04-24T23:13:56.804050+0000","last_undegraded":"2024-04-24T23:13:56.804050+0000","last_fullsized":"2024-04-24T23:13:56.804050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:15.358332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.338508+0000","last_change":"2024-04-24T23:12:26.994952+0000","last_active":"2024-04-24T23:13:28.338508+0000","last_peered":"2024-04-24T23:13:28.338508+0000","last_clean":"2024-04-24T23:13:28.338508+0000","last_became_active":"2024-04-24T23:12:26.994556+0000","last_became_peered":"2024-04-24T23:12:26.994556+0000","last_unstale":"2024-04-24T23:13:28.338508+0000","last_undegraded":"2024-04-24T23:13:28.338508+0000","last_fullsized":"2024-04-24T23:13:28.338508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.257427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953197+0000","last_change":"2024-04-24T23:11:16.721876+0000","last_active":"2024-04-24T23:13:27.953197+0000","last_peered":"2024-04-24T23:13:27.953197+0000","last_clean":"2024-04-24T23:13:27.953197+0000","last_became_active":"2024-04-24T23:11:16.721291+0000","last_became_peered":"2024-04-24T23:11:16.721291+0000","last_unstale":"2024-04-24T23:13:27.953197+0000","last_undegraded":"2024-04-24T23:13:27.953197+0000","last_fullsized":"2024-04-24T23:13:27.953197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_clean_scrub_stamp":"2024-04-24T23:07:51.257753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.458939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955980+0000","last_change":"2024-04-24T23:12:32.085905+0000","last_active":"2024-04-24T23:13:27.955980+0000","last_peered":"2024-04-24T23:13:27.955980+0000","last_clean":"2024-04-24T23:13:27.955980+0000","last_became_active":"2024-04-24T23:12:32.085374+0000","last_became_peered":"2024-04-24T23:12:32.085374+0000","last_unstale":"2024-04-24T23:13:27.955980+0000","last_undegraded":"2024-04-24T23:13:27.955980+0000","last_fullsized":"2024-04-24T23:13:27.955980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:16.290651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.806581+0000","last_change":"2024-04-24T23:12:27.999880+0000","last_active":"2024-04-24T23:13:56.806581+0000","last_peered":"2024-04-24T23:13:56.806581+0000","last_clean":"2024-04-24T23:13:56.806581+0000","last_became_active":"2024-04-24T23:12:27.999543+0000","last_became_peered":"2024-04-24T23:12:27.999543+0000","last_unstale":"2024-04-24T23:13:56.806581+0000","last_undegraded":"2024-04-24T23:13:56.806581+0000","last_fullsized":"2024-04-24T23:13:56.806581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:44.758968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.341009+0000","last_change":"2024-04-24T23:13:28.341009+0000","last_active":"2024-04-24T23:13:28.341009+0000","last_peered":"2024-04-24T23:13:28.341009+0000","last_clean":"2024-04-24T23:13:28.341009+0000","last_became_active":"2024-04-24T23:13:28.338192+0000","last_became_peered":"2024-04-24T23:13:28.338192+0000","last_unstale":"2024-04-24T23:13:28.341009+0000","last_undegraded":"2024-04-24T23:13:28.341009+0000","last_fullsized":"2024-04-24T23:13:28.341009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:11.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.954033+0000","last_change":"2024-04-24T23:12:32.024952+0000","last_active":"2024-04-24T23:13:27.954033+0000","last_peered":"2024-04-24T23:13:27.954033+0000","last_clean":"2024-04-24T23:13:27.954033+0000","last_became_active":"2024-04-24T23:12:32.024404+0000","last_became_peered":"2024-04-24T23:12:32.024404+0000","last_unstale":"2024-04-24T23:13:27.954033+0000","last_undegraded":"2024-04-24T23:13:27.954033+0000","last_fullsized":"2024-04-24T23:13:27.954033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.377794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952697+0000","last_change":"2024-04-24T23:12:27.007191+0000","last_active":"2024-04-24T23:13:27.952697+0000","last_peered":"2024-04-24T23:13:27.952697+0000","last_clean":"2024-04-24T23:13:27.952697+0000","last_became_active":"2024-04-24T23:12:27.006867+0000","last_became_peered":"2024-04-24T23:12:27.006867+0000","last_unstale":"2024-04-24T23:13:27.952697+0000","last_undegraded":"2024-04-24T23:13:27.952697+0000","last_fullsized":"2024-04-24T23:13:27.952697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:58.112293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.809060+0000","last_change":"2024-04-24T23:12:27.996322+0000","last_active":"2024-04-24T23:13:56.809060+0000","last_peered":"2024-04-24T23:13:56.809060+0000","last_clean":"2024-04-24T23:13:56.809060+0000","last_became_active":"2024-04-24T23:12:27.995991+0000","last_became_peered":"2024-04-24T23:12:27.995991+0000","last_unstale":"2024-04-24T23:13:56.809060+0000","last_undegraded":"2024-04-24T23:13:56.809060+0000","last_fullsized":"2024-04-24T23:13:56.809060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:08.960347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956492+0000","last_change":"2024-04-24T23:12:32.084959+0000","last_active":"2024-04-24T23:13:27.956492+0000","last_peered":"2024-04-24T23:13:27.956492+0000","last_clean":"2024-04-24T23:13:27.956492+0000","last_became_active":"2024-04-24T23:12:32.084553+0000","last_became_peered":"2024-04-24T23:12:32.084553+0000","last_unstale":"2024-04-24T23:13:27.956492+0000","last_undegraded":"2024-04-24T23:13:27.956492+0000","last_fullsized":"2024-04-24T23:13:27.956492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.836337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953483+0000","last_change":"2024-04-24T23:12:27.007742+0000","last_active":"2024-04-24T23:13:27.953483+0000","last_peered":"2024-04-24T23:13:27.953483+0000","last_clean":"2024-04-24T23:13:27.953483+0000","last_became_active":"2024-04-24T23:12:27.007423+0000","last_became_peered":"2024-04-24T23:12:27.007423+0000","last_unstale":"2024-04-24T23:13:27.953483+0000","last_undegraded":"2024-04-24T23:13:27.953483+0000","last_fullsized":"2024-04-24T23:13:27.953483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:39.478356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.804047+0000","last_change":"2024-04-24T23:12:28.010961+0000","last_active":"2024-04-24T23:13:56.804047+0000","last_peered":"2024-04-24T23:13:56.804047+0000","last_clean":"2024-04-24T23:13:56.804047+0000","last_became_active":"2024-04-24T23:12:28.010641+0000","last_became_peered":"2024-04-24T23:12:28.010641+0000","last_unstale":"2024-04-24T23:13:56.804047+0000","last_undegraded":"2024-04-24T23:13:56.804047+0000","last_fullsized":"2024-04-24T23:13:56.804047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:43.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956688+0000","last_change":"2024-04-24T23:12:32.023687+0000","last_active":"2024-04-24T23:13:27.956688+0000","last_peered":"2024-04-24T23:13:27.956688+0000","last_clean":"2024-04-24T23:13:27.956688+0000","last_became_active":"2024-04-24T23:12:32.023052+0000","last_became_peered":"2024-04-24T23:12:32.023052+0000","last_unstale":"2024-04-24T23:13:27.956688+0000","last_undegraded":"2024-04-24T23:13:27.956688+0000","last_fullsized":"2024-04-24T23:13:27.956688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:39.854030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.337090+0000","last_change":"2024-04-24T23:12:27.009743+0000","last_active":"2024-04-24T23:13:28.337090+0000","last_peered":"2024-04-24T23:13:28.337090+0000","last_clean":"2024-04-24T23:13:28.337090+0000","last_became_active":"2024-04-24T23:12:27.009343+0000","last_became_peered":"2024-04-24T23:12:27.009343+0000","last_unstale":"2024-04-24T23:13:28.337090+0000","last_undegraded":"2024-04-24T23:13:28.337090+0000","last_fullsized":"2024-04-24T23:13:28.337090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:20.387738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.818082+0000","last_change":"2024-04-24T23:13:27.961769+0000","last_active":"2024-04-24T23:13:56.818082+0000","last_peered":"2024-04-24T23:13:56.818082+0000","last_clean":"2024-04-24T23:13:56.818082+0000","last_became_active":"2024-04-24T23:13:27.961460+0000","last_became_peered":"2024-04-24T23:13:27.961460+0000","last_unstale":"2024-04-24T23:13:56.818082+0000","last_undegraded":"2024-04-24T23:13:56.818082+0000","last_fullsized":"2024-04-24T23:13:56.818082+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:01.012688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336604+0000","last_change":"2024-04-24T23:12:32.086209+0000","last_active":"2024-04-24T23:13:28.336604+0000","last_peered":"2024-04-24T23:13:28.336604+0000","last_clean":"2024-04-24T23:13:28.336604+0000","last_became_active":"2024-04-24T23:12:32.085752+0000","last_became_peered":"2024-04-24T23:12:32.085752+0000","last_unstale":"2024-04-24T23:13:28.336604+0000","last_undegraded":"2024-04-24T23:13:28.336604+0000","last_fullsized":"2024-04-24T23:13:28.336604+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:52.120905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.337370+0000","last_change":"2024-04-24T23:12:26.996355+0000","last_active":"2024-04-24T23:13:28.337370+0000","last_peered":"2024-04-24T23:13:28.337370+0000","last_clean":"2024-04-24T23:13:28.337370+0000","last_became_active":"2024-04-24T23:12:26.995486+0000","last_became_peered":"2024-04-24T23:12:26.995486+0000","last_unstale":"2024-04-24T23:13:28.337370+0000","last_undegraded":"2024-04-24T23:13:28.337370+0000","last_fullsized":"2024-04-24T23:13:28.337370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:48.981985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.794229+0000","last_change":"2024-04-24T23:12:28.004610+0000","last_active":"2024-04-24T23:13:56.794229+0000","last_peered":"2024-04-24T23:13:56.794229+0000","last_clean":"2024-04-24T23:13:56.794229+0000","last_became_active":"2024-04-24T23:12:28.004051+0000","last_became_peered":"2024-04-24T23:12:28.004051+0000","last_unstale":"2024-04-24T23:13:56.794229+0000","last_undegraded":"2024-04-24T23:13:56.794229+0000","last_fullsized":"2024-04-24T23:13:56.794229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:42.656954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953447+0000","last_change":"2024-04-24T23:12:32.029269+0000","last_active":"2024-04-24T23:13:27.953447+0000","last_peered":"2024-04-24T23:13:27.953447+0000","last_clean":"2024-04-24T23:13:27.953447+0000","last_became_active":"2024-04-24T23:12:32.028880+0000","last_became_peered":"2024-04-24T23:12:32.028880+0000","last_unstale":"2024-04-24T23:13:27.953447+0000","last_undegraded":"2024-04-24T23:13:27.953447+0000","last_fullsized":"2024-04-24T23:13:27.953447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:19.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956400+0000","last_change":"2024-04-24T23:12:27.006357+0000","last_active":"2024-04-24T23:13:27.956400+0000","last_peered":"2024-04-24T23:13:27.956400+0000","last_clean":"2024-04-24T23:13:27.956400+0000","last_became_active":"2024-04-24T23:12:27.006040+0000","last_became_peered":"2024-04-24T23:12:27.006040+0000","last_unstale":"2024-04-24T23:13:27.956400+0000","last_undegraded":"2024-04-24T23:13:27.956400+0000","last_fullsized":"2024-04-24T23:13:27.956400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:27.520663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.804128+0000","last_change":"2024-04-24T23:12:28.008909+0000","last_active":"2024-04-24T23:13:56.804128+0000","last_peered":"2024-04-24T23:13:56.804128+0000","last_clean":"2024-04-24T23:13:56.804128+0000","last_became_active":"2024-04-24T23:12:28.008552+0000","last_became_peered":"2024-04-24T23:12:28.008552+0000","last_unstale":"2024-04-24T23:13:56.804128+0000","last_undegraded":"2024-04-24T23:13:56.804128+0000","last_fullsized":"2024-04-24T23:13:56.804128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:20.628234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336135+0000","last_change":"2024-04-24T23:12:32.029720+0000","last_active":"2024-04-24T23:13:28.336135+0000","last_peered":"2024-04-24T23:13:28.336135+0000","last_clean":"2024-04-24T23:13:28.336135+0000","last_became_active":"2024-04-24T23:12:32.029311+0000","last_became_peered":"2024-04-24T23:12:32.029311+0000","last_unstale":"2024-04-24T23:13:28.336135+0000","last_undegraded":"2024-04-24T23:13:28.336135+0000","last_fullsized":"2024-04-24T23:13:28.336135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:37.832102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956975+0000","last_change":"2024-04-24T23:12:26.995552+0000","last_active":"2024-04-24T23:13:27.956975+0000","last_peered":"2024-04-24T23:13:27.956975+0000","last_clean":"2024-04-24T23:13:27.956975+0000","last_became_active":"2024-04-24T23:12:26.994922+0000","last_became_peered":"2024-04-24T23:12:26.994922+0000","last_unstale":"2024-04-24T23:13:27.956975+0000","last_undegraded":"2024-04-24T23:13:27.956975+0000","last_fullsized":"2024-04-24T23:13:27.956975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:17.981344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.803824+0000","last_change":"2024-04-24T23:12:28.008802+0000","last_active":"2024-04-24T23:13:56.803824+0000","last_peered":"2024-04-24T23:13:56.803824+0000","last_clean":"2024-04-24T23:13:56.803824+0000","last_became_active":"2024-04-24T23:12:28.008496+0000","last_became_peered":"2024-04-24T23:12:28.008496+0000","last_unstale":"2024-04-24T23:13:56.803824+0000","last_undegraded":"2024-04-24T23:13:56.803824+0000","last_fullsized":"2024-04-24T23:13:56.803824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:54.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336714+0000","last_change":"2024-04-24T23:12:32.085555+0000","last_active":"2024-04-24T23:13:28.336714+0000","last_peered":"2024-04-24T23:13:28.336714+0000","last_clean":"2024-04-24T23:13:28.336714+0000","last_became_active":"2024-04-24T23:12:32.085085+0000","last_became_peered":"2024-04-24T23:12:32.085085+0000","last_unstale":"2024-04-24T23:13:28.336714+0000","last_undegraded":"2024-04-24T23:13:28.336714+0000","last_fullsized":"2024-04-24T23:13:28.336714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:24.563964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.342962+0000","last_change":"2024-04-24T23:13:28.342962+0000","last_active":"2024-04-24T23:13:28.342962+0000","last_peered":"2024-04-24T23:13:28.342962+0000","last_clean":"2024-04-24T23:13:28.342962+0000","last_became_active":"2024-04-24T23:13:28.339742+0000","last_became_peered":"2024-04-24T23:13:28.339742+0000","last_unstale":"2024-04-24T23:13:28.342962+0000","last_undegraded":"2024-04-24T23:13:28.342962+0000","last_fullsized":"2024-04-24T23:13:28.342962+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:36.910863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820298+0000","last_change":"2024-04-24T23:12:28.014865+0000","last_active":"2024-04-24T23:13:56.820298+0000","last_peered":"2024-04-24T23:13:56.820298+0000","last_clean":"2024-04-24T23:13:56.820298+0000","last_became_active":"2024-04-24T23:12:28.014538+0000","last_became_peered":"2024-04-24T23:12:28.014538+0000","last_unstale":"2024-04-24T23:13:56.820298+0000","last_undegraded":"2024-04-24T23:13:56.820298+0000","last_fullsized":"2024-04-24T23:13:56.820298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:55.306883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336135+0000","last_change":"2024-04-24T23:12:32.024849+0000","last_active":"2024-04-24T23:13:28.336135+0000","last_peered":"2024-04-24T23:13:28.336135+0000","last_clean":"2024-04-24T23:13:28.336135+0000","last_became_active":"2024-04-24T23:12:32.024412+0000","last_became_peered":"2024-04-24T23:12:32.024412+0000","last_unstale":"2024-04-24T23:13:28.336135+0000","last_undegraded":"2024-04-24T23:13:28.336135+0000","last_fullsized":"2024-04-24T23:13:28.336135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:16.884565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":99,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:58.968692+0000","last_change":"2024-04-24T23:12:27.007284+0000","last_active":"2024-04-24T23:13:58.968692+0000","last_peered":"2024-04-24T23:13:58.968692+0000","last_clean":"2024-04-24T23:13:58.968692+0000","last_became_active":"2024-04-24T23:12:27.006916+0000","last_became_peered":"2024-04-24T23:12:27.006916+0000","last_unstale":"2024-04-24T23:13:58.968692+0000","last_undegraded":"2024-04-24T23:13:58.968692+0000","last_fullsized":"2024-04-24T23:13:58.968692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:40.333387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821395+0000","last_change":"2024-04-24T23:12:28.011603+0000","last_active":"2024-04-24T23:13:56.821395+0000","last_peered":"2024-04-24T23:13:56.821395+0000","last_clean":"2024-04-24T23:13:56.821395+0000","last_became_active":"2024-04-24T23:12:28.011268+0000","last_became_peered":"2024-04-24T23:12:28.011268+0000","last_unstale":"2024-04-24T23:13:56.821395+0000","last_undegraded":"2024-04-24T23:13:56.821395+0000","last_fullsized":"2024-04-24T23:13:56.821395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.693024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955139+0000","last_change":"2024-04-24T23:12:32.033965+0000","last_active":"2024-04-24T23:13:27.955139+0000","last_peered":"2024-04-24T23:13:27.955139+0000","last_clean":"2024-04-24T23:13:27.955139+0000","last_became_active":"2024-04-24T23:12:32.033472+0000","last_became_peered":"2024-04-24T23:12:32.033472+0000","last_unstale":"2024-04-24T23:13:27.955139+0000","last_undegraded":"2024-04-24T23:13:27.955139+0000","last_fullsized":"2024-04-24T23:13:27.955139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:05.243047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957238+0000","last_change":"2024-04-24T23:12:26.991981+0000","last_active":"2024-04-24T23:13:27.957238+0000","last_peered":"2024-04-24T23:13:27.957238+0000","last_clean":"2024-04-24T23:13:27.957238+0000","last_became_active":"2024-04-24T23:12:26.990707+0000","last_became_peered":"2024-04-24T23:12:26.990707+0000","last_unstale":"2024-04-24T23:13:27.957238+0000","last_undegraded":"2024-04-24T23:13:27.957238+0000","last_fullsized":"2024-04-24T23:13:27.957238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:36.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.817461+0000","last_change":"2024-04-24T23:12:28.010428+0000","last_active":"2024-04-24T23:13:56.817461+0000","last_peered":"2024-04-24T23:13:56.817461+0000","last_clean":"2024-04-24T23:13:56.817461+0000","last_became_active":"2024-04-24T23:12:28.010096+0000","last_became_peered":"2024-04-24T23:12:28.010096+0000","last_unstale":"2024-04-24T23:13:56.817461+0000","last_undegraded":"2024-04-24T23:13:56.817461+0000","last_fullsized":"2024-04-24T23:13:56.817461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:31.949617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955329+0000","last_change":"2024-04-24T23:12:32.023828+0000","last_active":"2024-04-24T23:13:27.955329+0000","last_peered":"2024-04-24T23:13:27.955329+0000","last_clean":"2024-04-24T23:13:27.955329+0000","last_became_active":"2024-04-24T23:12:32.023437+0000","last_became_peered":"2024-04-24T23:12:32.023437+0000","last_unstale":"2024-04-24T23:13:27.955329+0000","last_undegraded":"2024-04-24T23:13:27.955329+0000","last_fullsized":"2024-04-24T23:13:27.955329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.186860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989812+0000","last_change":"2024-04-24T23:12:26.993711+0000","last_active":"2024-04-24T23:13:27.989812+0000","last_peered":"2024-04-24T23:13:27.989812+0000","last_clean":"2024-04-24T23:13:27.989812+0000","last_became_active":"2024-04-24T23:12:26.993368+0000","last_became_peered":"2024-04-24T23:12:26.993368+0000","last_unstale":"2024-04-24T23:13:27.989812+0000","last_undegraded":"2024-04-24T23:13:27.989812+0000","last_fullsized":"2024-04-24T23:13:27.989812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:58.843747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.811933+0000","last_change":"2024-04-24T23:12:28.003026+0000","last_active":"2024-04-24T23:13:56.811933+0000","last_peered":"2024-04-24T23:13:56.811933+0000","last_clean":"2024-04-24T23:13:56.811933+0000","last_became_active":"2024-04-24T23:12:28.002693+0000","last_became_peered":"2024-04-24T23:12:28.002693+0000","last_unstale":"2024-04-24T23:13:56.811933+0000","last_undegraded":"2024-04-24T23:13:56.811933+0000","last_fullsized":"2024-04-24T23:13:56.811933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:30.926251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.807841+0000","last_change":"2024-04-24T23:12:28.013414+0000","last_active":"2024-04-24T23:13:56.807841+0000","last_peered":"2024-04-24T23:13:56.807841+0000","last_clean":"2024-04-24T23:13:56.807841+0000","last_became_active":"2024-04-24T23:12:28.013030+0000","last_became_peered":"2024-04-24T23:12:28.013030+0000","last_unstale":"2024-04-24T23:13:56.807841+0000","last_undegraded":"2024-04-24T23:13:56.807841+0000","last_fullsized":"2024-04-24T23:13:56.807841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:48.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.809798+0000","last_change":"2024-04-24T23:12:28.004821+0000","last_active":"2024-04-24T23:13:56.809798+0000","last_peered":"2024-04-24T23:13:56.809798+0000","last_clean":"2024-04-24T23:13:56.809798+0000","last_became_active":"2024-04-24T23:12:28.003162+0000","last_became_peered":"2024-04-24T23:12:28.003162+0000","last_unstale":"2024-04-24T23:13:56.809798+0000","last_undegraded":"2024-04-24T23:13:56.809798+0000","last_fullsized":"2024-04-24T23:13:56.809798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:59.109665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.809391+0000","last_change":"2024-04-24T23:12:28.002115+0000","last_active":"2024-04-24T23:13:56.809391+0000","last_peered":"2024-04-24T23:13:56.809391+0000","last_clean":"2024-04-24T23:13:56.809391+0000","last_became_active":"2024-04-24T23:12:28.001727+0000","last_became_peered":"2024-04-24T23:12:28.001727+0000","last_unstale":"2024-04-24T23:13:56.809391+0000","last_undegraded":"2024-04-24T23:13:56.809391+0000","last_fullsized":"2024-04-24T23:13:56.809391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:14.348843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.812005+0000","last_change":"2024-04-24T23:12:28.007784+0000","last_active":"2024-04-24T23:13:56.812005+0000","last_peered":"2024-04-24T23:13:56.812005+0000","last_clean":"2024-04-24T23:13:56.812005+0000","last_became_active":"2024-04-24T23:12:28.007481+0000","last_became_peered":"2024-04-24T23:12:28.007481+0000","last_unstale":"2024-04-24T23:13:56.812005+0000","last_undegraded":"2024-04-24T23:13:56.812005+0000","last_fullsized":"2024-04-24T23:13:56.812005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:33.753448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.803491+0000","last_change":"2024-04-24T23:12:28.009100+0000","last_active":"2024-04-24T23:13:56.803491+0000","last_peered":"2024-04-24T23:13:56.803491+0000","last_clean":"2024-04-24T23:13:56.803491+0000","last_became_active":"2024-04-24T23:12:28.008782+0000","last_became_peered":"2024-04-24T23:12:28.008782+0000","last_unstale":"2024-04-24T23:13:56.803491+0000","last_undegraded":"2024-04-24T23:13:56.803491+0000","last_fullsized":"2024-04-24T23:13:56.803491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:44.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.805972+0000","last_change":"2024-04-24T23:12:28.008129+0000","last_active":"2024-04-24T23:13:56.805972+0000","last_peered":"2024-04-24T23:13:56.805972+0000","last_clean":"2024-04-24T23:13:56.805972+0000","last_became_active":"2024-04-24T23:12:28.007819+0000","last_became_peered":"2024-04-24T23:12:28.007819+0000","last_unstale":"2024-04-24T23:13:56.805972+0000","last_undegraded":"2024-04-24T23:13:56.805972+0000","last_fullsized":"2024-04-24T23:13:56.805972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.234705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.805616+0000","last_change":"2024-04-24T23:12:27.993188+0000","last_active":"2024-04-24T23:13:56.805616+0000","last_peered":"2024-04-24T23:13:56.805616+0000","last_clean":"2024-04-24T23:13:56.805616+0000","last_became_active":"2024-04-24T23:12:27.992832+0000","last_became_peered":"2024-04-24T23:12:27.992832+0000","last_unstale":"2024-04-24T23:13:56.805616+0000","last_undegraded":"2024-04-24T23:13:56.805616+0000","last_fullsized":"2024-04-24T23:13:56.805616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:58.623164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.807479+0000","last_change":"2024-04-24T23:12:28.004099+0000","last_active":"2024-04-24T23:13:56.807479+0000","last_peered":"2024-04-24T23:13:56.807479+0000","last_clean":"2024-04-24T23:13:56.807479+0000","last_became_active":"2024-04-24T23:12:28.003802+0000","last_became_peered":"2024-04-24T23:12:28.003802+0000","last_unstale":"2024-04-24T23:13:56.807479+0000","last_undegraded":"2024-04-24T23:13:56.807479+0000","last_fullsized":"2024-04-24T23:13:56.807479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:25.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.806152+0000","last_change":"2024-04-24T23:12:28.007700+0000","last_active":"2024-04-24T23:13:56.806152+0000","last_peered":"2024-04-24T23:13:56.806152+0000","last_clean":"2024-04-24T23:13:56.806152+0000","last_became_active":"2024-04-24T23:12:28.002923+0000","last_became_peered":"2024-04-24T23:12:28.002923+0000","last_unstale":"2024-04-24T23:13:56.806152+0000","last_undegraded":"2024-04-24T23:13:56.806152+0000","last_fullsized":"2024-04-24T23:13:56.806152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:46.736096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.810863+0000","last_change":"2024-04-24T23:12:28.002194+0000","last_active":"2024-04-24T23:13:56.810863+0000","last_peered":"2024-04-24T23:13:56.810863+0000","last_clean":"2024-04-24T23:13:56.810863+0000","last_became_active":"2024-04-24T23:12:28.001703+0000","last_became_peered":"2024-04-24T23:12:28.001703+0000","last_unstale":"2024-04-24T23:13:56.810863+0000","last_undegraded":"2024-04-24T23:13:56.810863+0000","last_fullsized":"2024-04-24T23:13:56.810863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:20.887025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821672+0000","last_change":"2024-04-24T23:12:28.015980+0000","last_active":"2024-04-24T23:13:56.821672+0000","last_peered":"2024-04-24T23:13:56.821672+0000","last_clean":"2024-04-24T23:13:56.821672+0000","last_became_active":"2024-04-24T23:12:28.013510+0000","last_became_peered":"2024-04-24T23:12:28.013510+0000","last_unstale":"2024-04-24T23:13:56.821672+0000","last_undegraded":"2024-04-24T23:13:56.821672+0000","last_fullsized":"2024-04-24T23:13:56.821672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:37.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816253+0000","last_change":"2024-04-24T23:12:28.001883+0000","last_active":"2024-04-24T23:13:56.816253+0000","last_peered":"2024-04-24T23:13:56.816253+0000","last_clean":"2024-04-24T23:13:56.816253+0000","last_became_active":"2024-04-24T23:12:28.001499+0000","last_became_peered":"2024-04-24T23:12:28.001499+0000","last_unstale":"2024-04-24T23:13:56.816253+0000","last_undegraded":"2024-04-24T23:13:56.816253+0000","last_fullsized":"2024-04-24T23:13:56.816253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:46.805548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.803957+0000","last_change":"2024-04-24T23:12:28.004660+0000","last_active":"2024-04-24T23:13:56.803957+0000","last_peered":"2024-04-24T23:13:56.803957+0000","last_clean":"2024-04-24T23:13:56.803957+0000","last_became_active":"2024-04-24T23:12:28.004328+0000","last_became_peered":"2024-04-24T23:12:28.004328+0000","last_unstale":"2024-04-24T23:13:56.803957+0000","last_undegraded":"2024-04-24T23:13:56.803957+0000","last_fullsized":"2024-04-24T23:13:56.803957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:04.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70559,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":715,"num_read_kb":439,"num_write":248,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":506099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32876,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712276,"statfs":{"total":95995035648,"available":95961370624,"internally_reserved":0,"allocated":1740800,"data_stored":1156340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":509945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":506115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1102865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":501353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":565988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":502809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":566672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":501061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":505427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1671168,"data_stored":1098447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:14:06.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:14:07.173 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:07 smithi187 ceph-mon[33292]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:07 smithi187 ceph-mon[33292]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:07 smithi112 ceph-mon[29186]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:07 smithi112 ceph-mon[29186]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:07 smithi150 ceph-mon[32247]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:07 smithi150 ceph-mon[32247]: pgmap v342: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 -- 172.21.15.112:0/176654510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 msgr2=0x7feab410f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 --2- 172.21.15.112:0/176654510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 0x7feab410f110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feaac00b0d0 tx=0x7feaac00b3e0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 -- 172.21.15.112:0/176654510 shutdown_connections 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 --2- 172.21.15.112:0/176654510 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feab4102550 0x7feab4102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 --2- 172.21.15.112:0/176654510 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 0x7feab410f110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 --2- 172.21.15.112:0/176654510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feab4102f00 0x7feab4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 -- 172.21.15.112:0/176654510 >> 172.21.15.112:0/176654510 conn(0x7feab40793f0 msgr2=0x7feab4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 -- 172.21.15.112:0/176654510 shutdown_connections 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 -- 172.21.15.112:0/176654510 wait complete. 2024-04-24T23:14:09.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 Processor -- start 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.018+0000 7feabc643700 1 -- start start 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feabc643700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feab4102550 0x7feab41a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feabc643700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feab4102f00 0x7feab41a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feabc643700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 0x7feab419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feabc643700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feab4075b30 con 0x7feab4107540 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feabc643700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feab40759b0 con 0x7feab4102550 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feabc643700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feab4075cb0 con 0x7feab4102f00 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 0x7feab419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 0x7feab419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36552/0 (socket says 172.21.15.112:36552) 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 -- 172.21.15.112:0/2984363402 learned_addr learned my addr 172.21.15.112:0/2984363402 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feaba3df700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feab4102550 0x7feab41a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feab9bde700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feab4102f00 0x7feab41a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:09.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 -- 172.21.15.112:0/2984363402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feab4102f00 msgr2=0x7feab41a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feab4102f00 0x7feab41a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 -- 172.21.15.112:0/2984363402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feab4102550 msgr2=0x7feab41a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feab4102550 0x7feab41a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 -- 172.21.15.112:0/2984363402 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab419b1d0 con 0x7feab4107540 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feab9bde700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feab4102f00 0x7feab41a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feababe0700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 0x7feab419aa70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7feaac007a50 tx=0x7feaac007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.019+0000 7feaab7fe700 1 -- 172.21.15.112:0/2984363402 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac01f070 con 0x7feab4107540 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.020+0000 7feaab7fe700 1 -- 172.21.15.112:0/2984363402 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feaac003c00 con 0x7feab4107540 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.020+0000 7feaab7fe700 1 -- 172.21.15.112:0/2984363402 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac01b7b0 con 0x7feab4107540 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.020+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab419b460 con 0x7feab4107540 2024-04-24T23:14:09.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.020+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab41a7780 con 0x7feab4107540 2024-04-24T23:14:09.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.021+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab4103940 con 0x7feab4107540 2024-04-24T23:14:09.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.022+0000 7feaab7fe700 1 -- 172.21.15.112:0/2984363402 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feaac022070 con 0x7feab4107540 2024-04-24T23:14:09.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.022+0000 7feaab7fe700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feaa00a2c20 0x7feaa00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:09.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.022+0000 7feaba3df700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feaa00a2c20 0x7feaa00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:09.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.022+0000 7feaab7fe700 1 -- 172.21.15.112:0/2984363402 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7feaac0d0310 con 0x7feab4107540 2024-04-24T23:14:09.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.023+0000 7feaba3df700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feaa00a2c20 0x7feaa00a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feab419bbb0 tx=0x7feab000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:09.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.027+0000 7feaab7fe700 1 -- 172.21.15.112:0/2984363402 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7feaac09d040 con 0x7feab4107540 2024-04-24T23:14:09.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.217+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feab4100870 con 0x7feaa00a2c20 2024-04-24T23:14:09.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.226+0000 7feaab7fe700 1 -- 172.21.15.112:0/2984363402 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340521 (secure 0 0 0) 0x7feab4100870 con 0x7feaa00a2c20 2024-04-24T23:14:09.233 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:09.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feaa00a2c20 msgr2=0x7feaa00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feaa00a2c20 0x7feaa00a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7feab419bbb0 tx=0x7feab000b410 comp rx=0 tx=0).stop 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 msgr2=0x7feab419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 0x7feab419aa70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7feaac007a50 tx=0x7feaac007a80 comp rx=0 tx=0).stop 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 shutdown_connections 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feaa00a2c20 0x7feaa00a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feab4102550 0x7feab41a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab4107540 0x7feab419aa70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 --2- 172.21.15.112:0/2984363402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feab4102f00 0x7feab41a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 >> 172.21.15.112:0/2984363402 conn(0x7feab40793f0 msgr2=0x7feab4100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 shutdown_connections 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:09.229+0000 7feabc643700 1 -- 172.21.15.112:0/2984363402 wait complete. 2024-04-24T23:14:09.236 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-24T23:14:09.909 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-04-24T23:14:08.810507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661161,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":891,"num_read_kb":573,"num_write":431,"num_write_kb":3085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384336,"kb_used_data":14384,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557488,"statfs":{"total":1151940427776,"available":1151546867712,"internally_reserved":0,"allocated":14729216,"data_stored":8023121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":72,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003812"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.819286+0000","last_change":"2024-04-24T23:12:28.013506+0000","last_active":"2024-04-24T23:13:56.819286+0000","last_peered":"2024-04-24T23:13:56.819286+0000","last_clean":"2024-04-24T23:13:56.819286+0000","last_became_active":"2024-04-24T23:12:28.012652+0000","last_became_peered":"2024-04-24T23:12:28.012652+0000","last_unstale":"2024-04-24T23:13:56.819286+0000","last_undegraded":"2024-04-24T23:13:56.819286+0000","last_fullsized":"2024-04-24T23:13:56.819286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:57.420465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816013+0000","last_change":"2024-04-24T23:12:28.004998+0000","last_active":"2024-04-24T23:13:56.816013+0000","last_peered":"2024-04-24T23:13:56.816013+0000","last_clean":"2024-04-24T23:13:56.816013+0000","last_became_active":"2024-04-24T23:12:28.004706+0000","last_became_peered":"2024-04-24T23:12:28.004706+0000","last_unstale":"2024-04-24T23:13:56.816013+0000","last_undegraded":"2024-04-24T23:13:56.816013+0000","last_fullsized":"2024-04-24T23:13:56.816013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.759515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821050+0000","last_change":"2024-04-24T23:12:28.019170+0000","last_active":"2024-04-24T23:13:56.821050+0000","last_peered":"2024-04-24T23:13:56.821050+0000","last_clean":"2024-04-24T23:13:56.821050+0000","last_became_active":"2024-04-24T23:12:28.018815+0000","last_became_peered":"2024-04-24T23:12:28.018815+0000","last_unstale":"2024-04-24T23:13:56.821050+0000","last_undegraded":"2024-04-24T23:13:56.821050+0000","last_fullsized":"2024-04-24T23:13:56.821050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:43.830598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.796500+0000","last_change":"2024-04-24T23:12:28.008662+0000","last_active":"2024-04-24T23:13:56.796500+0000","last_peered":"2024-04-24T23:13:56.796500+0000","last_clean":"2024-04-24T23:13:56.796500+0000","last_became_active":"2024-04-24T23:12:28.008282+0000","last_became_peered":"2024-04-24T23:12:28.008282+0000","last_unstale":"2024-04-24T23:13:56.796500+0000","last_undegraded":"2024-04-24T23:13:56.796500+0000","last_fullsized":"2024-04-24T23:13:56.796500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:17.558899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.802262+0000","last_change":"2024-04-24T23:12:28.005212+0000","last_active":"2024-04-24T23:13:56.802262+0000","last_peered":"2024-04-24T23:13:56.802262+0000","last_clean":"2024-04-24T23:13:56.802262+0000","last_became_active":"2024-04-24T23:12:28.004853+0000","last_became_peered":"2024-04-24T23:12:28.004853+0000","last_unstale":"2024-04-24T23:13:56.802262+0000","last_undegraded":"2024-04-24T23:13:56.802262+0000","last_fullsized":"2024-04-24T23:13:56.802262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:33.650922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.801858+0000","last_change":"2024-04-24T23:12:28.004546+0000","last_active":"2024-04-24T23:13:56.801858+0000","last_peered":"2024-04-24T23:13:56.801858+0000","last_clean":"2024-04-24T23:13:56.801858+0000","last_became_active":"2024-04-24T23:12:28.004194+0000","last_became_peered":"2024-04-24T23:12:28.004194+0000","last_unstale":"2024-04-24T23:13:56.801858+0000","last_undegraded":"2024-04-24T23:13:56.801858+0000","last_fullsized":"2024-04-24T23:13:56.801858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:57.772271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.817292+0000","last_change":"2024-04-24T23:12:28.009436+0000","last_active":"2024-04-24T23:13:56.817292+0000","last_peered":"2024-04-24T23:13:56.817292+0000","last_clean":"2024-04-24T23:13:56.817292+0000","last_became_active":"2024-04-24T23:12:28.008219+0000","last_became_peered":"2024-04-24T23:12:28.008219+0000","last_unstale":"2024-04-24T23:13:56.817292+0000","last_undegraded":"2024-04-24T23:13:56.817292+0000","last_fullsized":"2024-04-24T23:13:56.817292+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.808577+0000","last_change":"2024-04-24T23:13:27.957345+0000","last_active":"2024-04-24T23:13:56.808577+0000","last_peered":"2024-04-24T23:13:56.808577+0000","last_clean":"2024-04-24T23:13:56.808577+0000","last_became_active":"2024-04-24T23:13:27.956989+0000","last_became_peered":"2024-04-24T23:13:27.956989+0000","last_unstale":"2024-04-24T23:13:56.808577+0000","last_undegraded":"2024-04-24T23:13:56.808577+0000","last_fullsized":"2024-04-24T23:13:56.808577+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:42.120100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.818499+0000","last_change":"2024-04-24T23:12:28.004948+0000","last_active":"2024-04-24T23:13:56.818499+0000","last_peered":"2024-04-24T23:13:56.818499+0000","last_clean":"2024-04-24T23:13:56.818499+0000","last_became_active":"2024-04-24T23:12:28.004380+0000","last_became_peered":"2024-04-24T23:12:28.004380+0000","last_unstale":"2024-04-24T23:13:56.818499+0000","last_undegraded":"2024-04-24T23:13:56.818499+0000","last_fullsized":"2024-04-24T23:13:56.818499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:53.999210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.807670+0000","last_change":"2024-04-24T23:12:28.005192+0000","last_active":"2024-04-24T23:13:56.807670+0000","last_peered":"2024-04-24T23:13:56.807670+0000","last_clean":"2024-04-24T23:13:56.807670+0000","last_became_active":"2024-04-24T23:12:28.004860+0000","last_became_peered":"2024-04-24T23:12:28.004860+0000","last_unstale":"2024-04-24T23:13:56.807670+0000","last_undegraded":"2024-04-24T23:13:56.807670+0000","last_fullsized":"2024-04-24T23:13:56.807670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:21.874408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820929+0000","last_change":"2024-04-24T23:12:28.016098+0000","last_active":"2024-04-24T23:13:56.820929+0000","last_peered":"2024-04-24T23:13:56.820929+0000","last_clean":"2024-04-24T23:13:56.820929+0000","last_became_active":"2024-04-24T23:12:28.015736+0000","last_became_peered":"2024-04-24T23:12:28.015736+0000","last_unstale":"2024-04-24T23:13:56.820929+0000","last_undegraded":"2024-04-24T23:13:56.820929+0000","last_fullsized":"2024-04-24T23:13:56.820929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.071797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.794305+0000","last_change":"2024-04-24T23:12:28.010651+0000","last_active":"2024-04-24T23:13:56.794305+0000","last_peered":"2024-04-24T23:13:56.794305+0000","last_clean":"2024-04-24T23:13:56.794305+0000","last_became_active":"2024-04-24T23:12:28.009700+0000","last_became_peered":"2024-04-24T23:12:28.009700+0000","last_unstale":"2024-04-24T23:13:56.794305+0000","last_undegraded":"2024-04-24T23:13:56.794305+0000","last_fullsized":"2024-04-24T23:13:56.794305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:26.172386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.813453+0000","last_change":"2024-04-24T23:12:28.005743+0000","last_active":"2024-04-24T23:13:56.813453+0000","last_peered":"2024-04-24T23:13:56.813453+0000","last_clean":"2024-04-24T23:13:56.813453+0000","last_became_active":"2024-04-24T23:12:28.005450+0000","last_became_peered":"2024-04-24T23:12:28.005450+0000","last_unstale":"2024-04-24T23:13:56.813453+0000","last_undegraded":"2024-04-24T23:13:56.813453+0000","last_fullsized":"2024-04-24T23:13:56.813453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:14.858212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816039+0000","last_change":"2024-04-24T23:12:28.004140+0000","last_active":"2024-04-24T23:13:56.816039+0000","last_peered":"2024-04-24T23:13:56.816039+0000","last_clean":"2024-04-24T23:13:56.816039+0000","last_became_active":"2024-04-24T23:12:28.003794+0000","last_became_peered":"2024-04-24T23:12:28.003794+0000","last_unstale":"2024-04-24T23:13:56.816039+0000","last_undegraded":"2024-04-24T23:13:56.816039+0000","last_fullsized":"2024-04-24T23:13:56.816039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:02.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.813379+0000","last_change":"2024-04-24T23:12:28.006834+0000","last_active":"2024-04-24T23:13:56.813379+0000","last_peered":"2024-04-24T23:13:56.813379+0000","last_clean":"2024-04-24T23:13:56.813379+0000","last_became_active":"2024-04-24T23:12:28.006504+0000","last_became_peered":"2024-04-24T23:12:28.006504+0000","last_unstale":"2024-04-24T23:13:56.813379+0000","last_undegraded":"2024-04-24T23:13:56.813379+0000","last_fullsized":"2024-04-24T23:13:56.813379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:21.194464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821873+0000","last_change":"2024-04-24T23:12:28.013132+0000","last_active":"2024-04-24T23:13:56.821873+0000","last_peered":"2024-04-24T23:13:56.821873+0000","last_clean":"2024-04-24T23:13:56.821873+0000","last_became_active":"2024-04-24T23:12:28.012751+0000","last_became_peered":"2024-04-24T23:12:28.012751+0000","last_unstale":"2024-04-24T23:13:56.821873+0000","last_undegraded":"2024-04-24T23:13:56.821873+0000","last_fullsized":"2024-04-24T23:13:56.821873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.942985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820913+0000","last_change":"2024-04-24T23:12:28.013719+0000","last_active":"2024-04-24T23:13:56.820913+0000","last_peered":"2024-04-24T23:13:56.820913+0000","last_clean":"2024-04-24T23:13:56.820913+0000","last_became_active":"2024-04-24T23:12:28.013108+0000","last_became_peered":"2024-04-24T23:12:28.013108+0000","last_unstale":"2024-04-24T23:13:56.820913+0000","last_undegraded":"2024-04-24T23:13:56.820913+0000","last_fullsized":"2024-04-24T23:13:56.820913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:38.248867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.819003+0000","last_change":"2024-04-24T23:12:28.012386+0000","last_active":"2024-04-24T23:13:56.819003+0000","last_peered":"2024-04-24T23:13:56.819003+0000","last_clean":"2024-04-24T23:13:56.819003+0000","last_became_active":"2024-04-24T23:12:28.011340+0000","last_became_peered":"2024-04-24T23:12:28.011340+0000","last_unstale":"2024-04-24T23:13:56.819003+0000","last_undegraded":"2024-04-24T23:13:56.819003+0000","last_fullsized":"2024-04-24T23:13:56.819003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:48.115600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.807008+0000","last_change":"2024-04-24T23:12:28.004033+0000","last_active":"2024-04-24T23:13:56.807008+0000","last_peered":"2024-04-24T23:13:56.807008+0000","last_clean":"2024-04-24T23:13:56.807008+0000","last_became_active":"2024-04-24T23:12:28.003648+0000","last_became_peered":"2024-04-24T23:12:28.003648+0000","last_unstale":"2024-04-24T23:13:56.807008+0000","last_undegraded":"2024-04-24T23:13:56.807008+0000","last_fullsized":"2024-04-24T23:13:56.807008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:13.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956921+0000","last_change":"2024-04-24T23:12:32.026590+0000","last_active":"2024-04-24T23:13:27.956921+0000","last_peered":"2024-04-24T23:13:27.956921+0000","last_clean":"2024-04-24T23:13:27.956921+0000","last_became_active":"2024-04-24T23:12:32.026098+0000","last_became_peered":"2024-04-24T23:12:32.026098+0000","last_unstale":"2024-04-24T23:13:27.956921+0000","last_undegraded":"2024-04-24T23:13:27.956921+0000","last_fullsized":"2024-04-24T23:13:27.956921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:55.812641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":113,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:05.977719+0000","last_change":"2024-04-24T23:12:27.005325+0000","last_active":"2024-04-24T23:14:05.977719+0000","last_peered":"2024-04-24T23:14:05.977719+0000","last_clean":"2024-04-24T23:14:05.977719+0000","last_became_active":"2024-04-24T23:12:27.004627+0000","last_became_peered":"2024-04-24T23:12:27.004627+0000","last_unstale":"2024-04-24T23:14:05.977719+0000","last_undegraded":"2024-04-24T23:14:05.977719+0000","last_fullsized":"2024-04-24T23:14:05.977719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.985647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.809403+0000","last_change":"2024-04-24T23:12:28.007872+0000","last_active":"2024-04-24T23:13:56.809403+0000","last_peered":"2024-04-24T23:13:56.809403+0000","last_clean":"2024-04-24T23:13:56.809403+0000","last_became_active":"2024-04-24T23:12:28.007493+0000","last_became_peered":"2024-04-24T23:12:28.007493+0000","last_unstale":"2024-04-24T23:13:56.809403+0000","last_undegraded":"2024-04-24T23:13:56.809403+0000","last_fullsized":"2024-04-24T23:13:56.809403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:26.686831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336135+0000","last_change":"2024-04-24T23:12:32.031138+0000","last_active":"2024-04-24T23:13:28.336135+0000","last_peered":"2024-04-24T23:13:28.336135+0000","last_clean":"2024-04-24T23:13:28.336135+0000","last_became_active":"2024-04-24T23:12:32.030626+0000","last_became_peered":"2024-04-24T23:12:32.030626+0000","last_unstale":"2024-04-24T23:13:28.336135+0000","last_undegraded":"2024-04-24T23:13:28.336135+0000","last_fullsized":"2024-04-24T23:13:28.336135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957180+0000","last_change":"2024-04-24T23:12:26.999954+0000","last_active":"2024-04-24T23:13:27.957180+0000","last_peered":"2024-04-24T23:13:27.957180+0000","last_clean":"2024-04-24T23:13:27.957180+0000","last_became_active":"2024-04-24T23:12:26.999507+0000","last_became_peered":"2024-04-24T23:12:26.999507+0000","last_unstale":"2024-04-24T23:13:27.957180+0000","last_undegraded":"2024-04-24T23:13:27.957180+0000","last_fullsized":"2024-04-24T23:13:27.957180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:23.984002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821143+0000","last_change":"2024-04-24T23:12:28.019342+0000","last_active":"2024-04-24T23:13:56.821143+0000","last_peered":"2024-04-24T23:13:56.821143+0000","last_clean":"2024-04-24T23:13:56.821143+0000","last_became_active":"2024-04-24T23:12:28.018976+0000","last_became_peered":"2024-04-24T23:12:28.018976+0000","last_unstale":"2024-04-24T23:13:56.821143+0000","last_undegraded":"2024-04-24T23:13:56.821143+0000","last_fullsized":"2024-04-24T23:13:56.821143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:18.737495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.337245+0000","last_change":"2024-04-24T23:12:32.031015+0000","last_active":"2024-04-24T23:13:28.337245+0000","last_peered":"2024-04-24T23:13:28.337245+0000","last_clean":"2024-04-24T23:13:28.337245+0000","last_became_active":"2024-04-24T23:12:32.030564+0000","last_became_peered":"2024-04-24T23:12:32.030564+0000","last_unstale":"2024-04-24T23:13:28.337245+0000","last_undegraded":"2024-04-24T23:13:28.337245+0000","last_fullsized":"2024-04-24T23:13:28.337245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:12.247397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952539+0000","last_change":"2024-04-24T23:12:27.001229+0000","last_active":"2024-04-24T23:13:27.952539+0000","last_peered":"2024-04-24T23:13:27.952539+0000","last_clean":"2024-04-24T23:13:27.952539+0000","last_became_active":"2024-04-24T23:12:27.000889+0000","last_became_peered":"2024-04-24T23:12:27.000889+0000","last_unstale":"2024-04-24T23:13:27.952539+0000","last_undegraded":"2024-04-24T23:13:27.952539+0000","last_fullsized":"2024-04-24T23:13:27.952539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:39.189828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820422+0000","last_change":"2024-04-24T23:12:28.012309+0000","last_active":"2024-04-24T23:13:56.820422+0000","last_peered":"2024-04-24T23:13:56.820422+0000","last_clean":"2024-04-24T23:13:56.820422+0000","last_became_active":"2024-04-24T23:12:28.011537+0000","last_became_peered":"2024-04-24T23:12:28.011537+0000","last_unstale":"2024-04-24T23:13:56.820422+0000","last_undegraded":"2024-04-24T23:13:56.820422+0000","last_fullsized":"2024-04-24T23:13:56.820422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:38.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336611+0000","last_change":"2024-04-24T23:12:32.030830+0000","last_active":"2024-04-24T23:13:28.336611+0000","last_peered":"2024-04-24T23:13:28.336611+0000","last_clean":"2024-04-24T23:13:28.336611+0000","last_became_active":"2024-04-24T23:12:32.030318+0000","last_became_peered":"2024-04-24T23:12:32.030318+0000","last_unstale":"2024-04-24T23:13:28.336611+0000","last_undegraded":"2024-04-24T23:13:28.336611+0000","last_fullsized":"2024-04-24T23:13:28.336611+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:25.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952080+0000","last_change":"2024-04-24T23:12:26.997926+0000","last_active":"2024-04-24T23:13:27.952080+0000","last_peered":"2024-04-24T23:13:27.952080+0000","last_clean":"2024-04-24T23:13:27.952080+0000","last_became_active":"2024-04-24T23:12:26.997552+0000","last_became_peered":"2024-04-24T23:12:26.997552+0000","last_unstale":"2024-04-24T23:13:27.952080+0000","last_undegraded":"2024-04-24T23:13:27.952080+0000","last_fullsized":"2024-04-24T23:13:27.952080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:50.926624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816520+0000","last_change":"2024-04-24T23:13:28.338152+0000","last_active":"2024-04-24T23:13:56.816520+0000","last_peered":"2024-04-24T23:13:56.816520+0000","last_clean":"2024-04-24T23:13:56.816520+0000","last_became_active":"2024-04-24T23:13:28.337728+0000","last_became_peered":"2024-04-24T23:13:28.337728+0000","last_unstale":"2024-04-24T23:13:56.816520+0000","last_undegraded":"2024-04-24T23:13:56.816520+0000","last_fullsized":"2024-04-24T23:13:56.816520+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.179108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957588+0000","last_change":"2024-04-24T23:12:32.021601+0000","last_active":"2024-04-24T23:13:27.957588+0000","last_peered":"2024-04-24T23:13:27.957588+0000","last_clean":"2024-04-24T23:13:27.957588+0000","last_became_active":"2024-04-24T23:12:32.021141+0000","last_became_peered":"2024-04-24T23:12:32.021141+0000","last_unstale":"2024-04-24T23:13:27.957588+0000","last_undegraded":"2024-04-24T23:13:27.957588+0000","last_fullsized":"2024-04-24T23:13:27.957588+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:53.795337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":110,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:03.976556+0000","last_change":"2024-04-24T23:12:27.002639+0000","last_active":"2024-04-24T23:14:03.976556+0000","last_peered":"2024-04-24T23:14:03.976556+0000","last_clean":"2024-04-24T23:14:03.976556+0000","last_became_active":"2024-04-24T23:12:27.002295+0000","last_became_peered":"2024-04-24T23:12:27.002295+0000","last_unstale":"2024-04-24T23:14:03.976556+0000","last_undegraded":"2024-04-24T23:14:03.976556+0000","last_fullsized":"2024-04-24T23:14:03.976556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:40.962878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.801867+0000","last_change":"2024-04-24T23:12:28.000123+0000","last_active":"2024-04-24T23:13:56.801867+0000","last_peered":"2024-04-24T23:13:56.801867+0000","last_clean":"2024-04-24T23:13:56.801867+0000","last_became_active":"2024-04-24T23:12:27.999778+0000","last_became_peered":"2024-04-24T23:12:27.999778+0000","last_unstale":"2024-04-24T23:13:56.801867+0000","last_undegraded":"2024-04-24T23:13:56.801867+0000","last_fullsized":"2024-04-24T23:13:56.801867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:03.827336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957374+0000","last_change":"2024-04-24T23:12:32.023886+0000","last_active":"2024-04-24T23:13:27.957374+0000","last_peered":"2024-04-24T23:13:27.957374+0000","last_clean":"2024-04-24T23:13:27.957374+0000","last_became_active":"2024-04-24T23:12:32.022796+0000","last_became_peered":"2024-04-24T23:12:32.022796+0000","last_unstale":"2024-04-24T23:13:27.957374+0000","last_undegraded":"2024-04-24T23:13:27.957374+0000","last_fullsized":"2024-04-24T23:13:27.957374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:17.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.951446+0000","last_change":"2024-04-24T23:12:26.999210+0000","last_active":"2024-04-24T23:13:27.951446+0000","last_peered":"2024-04-24T23:13:27.951446+0000","last_clean":"2024-04-24T23:13:27.951446+0000","last_became_active":"2024-04-24T23:12:26.998642+0000","last_became_peered":"2024-04-24T23:12:26.998642+0000","last_unstale":"2024-04-24T23:13:27.951446+0000","last_undegraded":"2024-04-24T23:13:27.951446+0000","last_fullsized":"2024-04-24T23:13:27.951446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:39.171955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.812746+0000","last_change":"2024-04-24T23:13:27.990737+0000","last_active":"2024-04-24T23:13:56.812746+0000","last_peered":"2024-04-24T23:13:56.812746+0000","last_clean":"2024-04-24T23:13:56.812746+0000","last_became_active":"2024-04-24T23:13:27.990428+0000","last_became_peered":"2024-04-24T23:13:27.990428+0000","last_unstale":"2024-04-24T23:13:56.812746+0000","last_undegraded":"2024-04-24T23:13:56.812746+0000","last_fullsized":"2024-04-24T23:13:56.812746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:40.163241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.954317+0000","last_change":"2024-04-24T23:12:32.028192+0000","last_active":"2024-04-24T23:13:27.954317+0000","last_peered":"2024-04-24T23:13:27.954317+0000","last_clean":"2024-04-24T23:13:27.954317+0000","last_became_active":"2024-04-24T23:12:32.027715+0000","last_became_peered":"2024-04-24T23:12:32.027715+0000","last_unstale":"2024-04-24T23:13:27.954317+0000","last_undegraded":"2024-04-24T23:13:27.954317+0000","last_fullsized":"2024-04-24T23:13:27.954317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:22.577980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989453+0000","last_change":"2024-04-24T23:12:27.000212+0000","last_active":"2024-04-24T23:13:27.989453+0000","last_peered":"2024-04-24T23:13:27.989453+0000","last_clean":"2024-04-24T23:13:27.989453+0000","last_became_active":"2024-04-24T23:12:26.999844+0000","last_became_peered":"2024-04-24T23:12:26.999844+0000","last_unstale":"2024-04-24T23:13:27.989453+0000","last_undegraded":"2024-04-24T23:13:27.989453+0000","last_fullsized":"2024-04-24T23:13:27.989453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:00.837670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.804402+0000","last_change":"2024-04-24T23:12:28.002875+0000","last_active":"2024-04-24T23:13:56.804402+0000","last_peered":"2024-04-24T23:13:56.804402+0000","last_clean":"2024-04-24T23:13:56.804402+0000","last_became_active":"2024-04-24T23:12:28.002572+0000","last_became_peered":"2024-04-24T23:12:28.002572+0000","last_unstale":"2024-04-24T23:13:56.804402+0000","last_undegraded":"2024-04-24T23:13:56.804402+0000","last_fullsized":"2024-04-24T23:13:56.804402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:55.931371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955452+0000","last_change":"2024-04-24T23:12:32.024729+0000","last_active":"2024-04-24T23:13:27.955452+0000","last_peered":"2024-04-24T23:13:27.955452+0000","last_clean":"2024-04-24T23:13:27.955452+0000","last_became_active":"2024-04-24T23:12:32.024316+0000","last_became_peered":"2024-04-24T23:12:32.024316+0000","last_unstale":"2024-04-24T23:13:27.955452+0000","last_undegraded":"2024-04-24T23:13:27.955452+0000","last_fullsized":"2024-04-24T23:13:27.955452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:21.668782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955592+0000","last_change":"2024-04-24T23:12:27.003435+0000","last_active":"2024-04-24T23:13:27.955592+0000","last_peered":"2024-04-24T23:13:27.955592+0000","last_clean":"2024-04-24T23:13:27.955592+0000","last_became_active":"2024-04-24T23:12:27.003101+0000","last_became_peered":"2024-04-24T23:12:27.003101+0000","last_unstale":"2024-04-24T23:13:27.955592+0000","last_undegraded":"2024-04-24T23:13:27.955592+0000","last_fullsized":"2024-04-24T23:13:27.955592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:42.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.808426+0000","last_change":"2024-04-24T23:12:28.005056+0000","last_active":"2024-04-24T23:13:56.808426+0000","last_peered":"2024-04-24T23:13:56.808426+0000","last_clean":"2024-04-24T23:13:56.808426+0000","last_became_active":"2024-04-24T23:12:28.004725+0000","last_became_peered":"2024-04-24T23:12:28.004725+0000","last_unstale":"2024-04-24T23:13:56.808426+0000","last_undegraded":"2024-04-24T23:13:56.808426+0000","last_fullsized":"2024-04-24T23:13:56.808426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:49.384950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956393+0000","last_change":"2024-04-24T23:12:32.032947+0000","last_active":"2024-04-24T23:13:27.956393+0000","last_peered":"2024-04-24T23:13:27.956393+0000","last_clean":"2024-04-24T23:13:27.956393+0000","last_became_active":"2024-04-24T23:12:32.032560+0000","last_became_peered":"2024-04-24T23:12:32.032560+0000","last_unstale":"2024-04-24T23:13:27.956393+0000","last_undegraded":"2024-04-24T23:13:27.956393+0000","last_fullsized":"2024-04-24T23:13:27.956393+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:29.282938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":180,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:07.981309+0000","last_change":"2024-04-24T23:12:27.000011+0000","last_active":"2024-04-24T23:14:07.981309+0000","last_peered":"2024-04-24T23:14:07.981309+0000","last_clean":"2024-04-24T23:14:07.981309+0000","last_became_active":"2024-04-24T23:12:26.999572+0000","last_became_peered":"2024-04-24T23:12:26.999572+0000","last_unstale":"2024-04-24T23:14:07.981309+0000","last_undegraded":"2024-04-24T23:14:07.981309+0000","last_fullsized":"2024-04-24T23:14:07.981309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:50.453964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820896+0000","last_change":"2024-04-24T23:12:28.012808+0000","last_active":"2024-04-24T23:13:56.820896+0000","last_peered":"2024-04-24T23:13:56.820896+0000","last_clean":"2024-04-24T23:13:56.820896+0000","last_became_active":"2024-04-24T23:12:28.012406+0000","last_became_peered":"2024-04-24T23:12:28.012406+0000","last_unstale":"2024-04-24T23:13:56.820896+0000","last_undegraded":"2024-04-24T23:13:56.820896+0000","last_fullsized":"2024-04-24T23:13:56.820896+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:19.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953445+0000","last_change":"2024-04-24T23:12:32.027156+0000","last_active":"2024-04-24T23:13:27.953445+0000","last_peered":"2024-04-24T23:13:27.953445+0000","last_clean":"2024-04-24T23:13:27.953445+0000","last_became_active":"2024-04-24T23:12:32.026743+0000","last_became_peered":"2024-04-24T23:12:32.026743+0000","last_unstale":"2024-04-24T23:13:27.953445+0000","last_undegraded":"2024-04-24T23:13:27.953445+0000","last_fullsized":"2024-04-24T23:13:27.953445+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:43.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953121+0000","last_change":"2024-04-24T23:12:27.001832+0000","last_active":"2024-04-24T23:13:27.953121+0000","last_peered":"2024-04-24T23:13:27.953121+0000","last_clean":"2024-04-24T23:13:27.953121+0000","last_became_active":"2024-04-24T23:12:27.001482+0000","last_became_peered":"2024-04-24T23:12:27.001482+0000","last_unstale":"2024-04-24T23:13:27.953121+0000","last_undegraded":"2024-04-24T23:13:27.953121+0000","last_fullsized":"2024-04-24T23:13:27.953121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:28.949767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.813438+0000","last_change":"2024-04-24T23:13:27.960551+0000","last_active":"2024-04-24T23:13:56.813438+0000","last_peered":"2024-04-24T23:13:56.813438+0000","last_clean":"2024-04-24T23:13:56.813438+0000","last_became_active":"2024-04-24T23:13:27.960264+0000","last_became_peered":"2024-04-24T23:13:27.960264+0000","last_unstale":"2024-04-24T23:13:56.813438+0000","last_undegraded":"2024-04-24T23:13:56.813438+0000","last_fullsized":"2024-04-24T23:13:56.813438+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:38.758688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.337104+0000","last_change":"2024-04-24T23:12:32.023007+0000","last_active":"2024-04-24T23:13:28.337104+0000","last_peered":"2024-04-24T23:13:28.337104+0000","last_clean":"2024-04-24T23:13:28.337104+0000","last_became_active":"2024-04-24T23:12:32.021324+0000","last_became_peered":"2024-04-24T23:12:32.021324+0000","last_unstale":"2024-04-24T23:13:28.337104+0000","last_undegraded":"2024-04-24T23:13:28.337104+0000","last_fullsized":"2024-04-24T23:13:28.337104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:41.542778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956673+0000","last_change":"2024-04-24T23:12:26.990562+0000","last_active":"2024-04-24T23:13:27.956673+0000","last_peered":"2024-04-24T23:13:27.956673+0000","last_clean":"2024-04-24T23:13:27.956673+0000","last_became_active":"2024-04-24T23:12:26.990146+0000","last_became_peered":"2024-04-24T23:12:26.990146+0000","last_unstale":"2024-04-24T23:13:27.956673+0000","last_undegraded":"2024-04-24T23:13:27.956673+0000","last_fullsized":"2024-04-24T23:13:27.956673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.621051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.811856+0000","last_change":"2024-04-24T23:12:28.000954+0000","last_active":"2024-04-24T23:13:56.811856+0000","last_peered":"2024-04-24T23:13:56.811856+0000","last_clean":"2024-04-24T23:13:56.811856+0000","last_became_active":"2024-04-24T23:12:28.000610+0000","last_became_peered":"2024-04-24T23:12:28.000610+0000","last_unstale":"2024-04-24T23:13:56.811856+0000","last_undegraded":"2024-04-24T23:13:56.811856+0000","last_fullsized":"2024-04-24T23:13:56.811856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:32.585919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952932+0000","last_change":"2024-04-24T23:12:32.030373+0000","last_active":"2024-04-24T23:13:27.952932+0000","last_peered":"2024-04-24T23:13:27.952932+0000","last_clean":"2024-04-24T23:13:27.952932+0000","last_became_active":"2024-04-24T23:12:32.029954+0000","last_became_peered":"2024-04-24T23:12:32.029954+0000","last_unstale":"2024-04-24T23:13:27.952932+0000","last_undegraded":"2024-04-24T23:13:27.952932+0000","last_fullsized":"2024-04-24T23:13:27.952932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:46:28.061509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":224,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:07.981319+0000","last_change":"2024-04-24T23:12:26.996719+0000","last_active":"2024-04-24T23:14:07.981319+0000","last_peered":"2024-04-24T23:14:07.981319+0000","last_clean":"2024-04-24T23:14:07.981319+0000","last_became_active":"2024-04-24T23:12:26.996360+0000","last_became_peered":"2024-04-24T23:12:26.996360+0000","last_unstale":"2024-04-24T23:14:07.981319+0000","last_undegraded":"2024-04-24T23:14:07.981319+0000","last_fullsized":"2024-04-24T23:14:07.981319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.526902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":159,"num_read_kb":142,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.799339+0000","last_change":"2024-04-24T23:12:28.002889+0000","last_active":"2024-04-24T23:13:56.799339+0000","last_peered":"2024-04-24T23:13:56.799339+0000","last_clean":"2024-04-24T23:13:56.799339+0000","last_became_active":"2024-04-24T23:12:28.002540+0000","last_became_peered":"2024-04-24T23:12:28.002540+0000","last_unstale":"2024-04-24T23:13:56.799339+0000","last_undegraded":"2024-04-24T23:13:56.799339+0000","last_fullsized":"2024-04-24T23:13:56.799339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:57.792628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957499+0000","last_change":"2024-04-24T23:12:32.031160+0000","last_active":"2024-04-24T23:13:27.957499+0000","last_peered":"2024-04-24T23:13:27.957499+0000","last_clean":"2024-04-24T23:13:27.957499+0000","last_became_active":"2024-04-24T23:12:32.030719+0000","last_became_peered":"2024-04-24T23:12:32.030719+0000","last_unstale":"2024-04-24T23:13:27.957499+0000","last_undegraded":"2024-04-24T23:13:27.957499+0000","last_fullsized":"2024-04-24T23:13:27.957499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:45.648011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.967634+0000","last_change":"2024-04-24T23:13:27.967294+0000","last_active":"2024-04-24T23:13:27.967634+0000","last_peered":"2024-04-24T23:13:27.967634+0000","last_clean":"2024-04-24T23:13:27.967634+0000","last_became_active":"2024-04-24T23:13:27.958064+0000","last_became_peered":"2024-04-24T23:13:27.958064+0000","last_unstale":"2024-04-24T23:13:27.967634+0000","last_undegraded":"2024-04-24T23:13:27.967634+0000","last_fullsized":"2024-04-24T23:13:27.967634+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:08.824502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816390+0000","last_change":"2024-04-24T23:12:28.003211+0000","last_active":"2024-04-24T23:13:56.816390+0000","last_peered":"2024-04-24T23:13:56.816390+0000","last_clean":"2024-04-24T23:13:56.816390+0000","last_became_active":"2024-04-24T23:12:28.002897+0000","last_became_peered":"2024-04-24T23:12:28.002897+0000","last_unstale":"2024-04-24T23:13:56.816390+0000","last_undegraded":"2024-04-24T23:13:56.816390+0000","last_fullsized":"2024-04-24T23:13:56.816390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.187239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953805+0000","last_change":"2024-04-24T23:12:32.028857+0000","last_active":"2024-04-24T23:13:27.953805+0000","last_peered":"2024-04-24T23:13:27.953805+0000","last_clean":"2024-04-24T23:13:27.953805+0000","last_became_active":"2024-04-24T23:12:32.028455+0000","last_became_peered":"2024-04-24T23:12:32.028455+0000","last_unstale":"2024-04-24T23:13:27.953805+0000","last_undegraded":"2024-04-24T23:13:27.953805+0000","last_fullsized":"2024-04-24T23:13:27.953805+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:23.073895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955506+0000","last_change":"2024-04-24T23:12:27.003525+0000","last_active":"2024-04-24T23:13:27.955506+0000","last_peered":"2024-04-24T23:13:27.955506+0000","last_clean":"2024-04-24T23:13:27.955506+0000","last_became_active":"2024-04-24T23:12:27.003203+0000","last_became_peered":"2024-04-24T23:12:27.003203+0000","last_unstale":"2024-04-24T23:13:27.955506+0000","last_undegraded":"2024-04-24T23:13:27.955506+0000","last_fullsized":"2024-04-24T23:13:27.955506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:18.521925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.814230+0000","last_change":"2024-04-24T23:12:28.017111+0000","last_active":"2024-04-24T23:13:56.814230+0000","last_peered":"2024-04-24T23:13:56.814230+0000","last_clean":"2024-04-24T23:13:56.814230+0000","last_became_active":"2024-04-24T23:12:28.016725+0000","last_became_peered":"2024-04-24T23:12:28.016725+0000","last_unstale":"2024-04-24T23:13:56.814230+0000","last_undegraded":"2024-04-24T23:13:56.814230+0000","last_fullsized":"2024-04-24T23:13:56.814230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:25.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953210+0000","last_change":"2024-04-24T23:12:32.027963+0000","last_active":"2024-04-24T23:13:27.953210+0000","last_peered":"2024-04-24T23:13:27.953210+0000","last_clean":"2024-04-24T23:13:27.953210+0000","last_became_active":"2024-04-24T23:12:32.027585+0000","last_became_peered":"2024-04-24T23:12:32.027585+0000","last_unstale":"2024-04-24T23:13:27.953210+0000","last_undegraded":"2024-04-24T23:13:27.953210+0000","last_fullsized":"2024-04-24T23:13:27.953210+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:43:27.111414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":106,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:03.971216+0000","last_change":"2024-04-24T23:12:27.002797+0000","last_active":"2024-04-24T23:14:03.971216+0000","last_peered":"2024-04-24T23:14:03.971216+0000","last_clean":"2024-04-24T23:14:03.971216+0000","last_became_active":"2024-04-24T23:12:27.002086+0000","last_became_peered":"2024-04-24T23:12:27.002086+0000","last_unstale":"2024-04-24T23:14:03.971216+0000","last_undegraded":"2024-04-24T23:14:03.971216+0000","last_fullsized":"2024-04-24T23:14:03.971216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:55.025453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.815909+0000","last_change":"2024-04-24T23:13:27.955130+0000","last_active":"2024-04-24T23:13:56.815909+0000","last_peered":"2024-04-24T23:13:56.815909+0000","last_clean":"2024-04-24T23:13:56.815909+0000","last_became_active":"2024-04-24T23:13:27.954784+0000","last_became_peered":"2024-04-24T23:13:27.954784+0000","last_unstale":"2024-04-24T23:13:56.815909+0000","last_undegraded":"2024-04-24T23:13:56.815909+0000","last_fullsized":"2024-04-24T23:13:56.815909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:10.574771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989872+0000","last_change":"2024-04-24T23:12:32.029104+0000","last_active":"2024-04-24T23:13:27.989872+0000","last_peered":"2024-04-24T23:13:27.989872+0000","last_clean":"2024-04-24T23:13:27.989872+0000","last_became_active":"2024-04-24T23:12:32.028704+0000","last_became_peered":"2024-04-24T23:12:32.028704+0000","last_unstale":"2024-04-24T23:13:27.989872+0000","last_undegraded":"2024-04-24T23:13:27.989872+0000","last_fullsized":"2024-04-24T23:13:27.989872+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:33.200884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.977205+0000","last_change":"2024-04-24T23:13:27.976835+0000","last_active":"2024-04-24T23:13:27.977205+0000","last_peered":"2024-04-24T23:13:27.977205+0000","last_clean":"2024-04-24T23:13:27.977205+0000","last_became_active":"2024-04-24T23:13:27.959684+0000","last_became_peered":"2024-04-24T23:13:27.959684+0000","last_unstale":"2024-04-24T23:13:27.977205+0000","last_undegraded":"2024-04-24T23:13:27.977205+0000","last_fullsized":"2024-04-24T23:13:27.977205+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:24.327964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.811557+0000","last_change":"2024-04-24T23:12:28.010510+0000","last_active":"2024-04-24T23:13:56.811557+0000","last_peered":"2024-04-24T23:13:56.811557+0000","last_clean":"2024-04-24T23:13:56.811557+0000","last_became_active":"2024-04-24T23:12:28.009244+0000","last_became_peered":"2024-04-24T23:12:28.009244+0000","last_unstale":"2024-04-24T23:13:56.811557+0000","last_undegraded":"2024-04-24T23:13:56.811557+0000","last_fullsized":"2024-04-24T23:13:56.811557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.146563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955268+0000","last_change":"2024-04-24T23:12:32.031156+0000","last_active":"2024-04-24T23:13:27.955268+0000","last_peered":"2024-04-24T23:13:27.955268+0000","last_clean":"2024-04-24T23:13:27.955268+0000","last_became_active":"2024-04-24T23:12:32.030653+0000","last_became_peered":"2024-04-24T23:12:32.030653+0000","last_unstale":"2024-04-24T23:13:27.955268+0000","last_undegraded":"2024-04-24T23:13:27.955268+0000","last_fullsized":"2024-04-24T23:13:27.955268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:48.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952365+0000","last_change":"2024-04-24T23:12:26.998003+0000","last_active":"2024-04-24T23:13:27.952365+0000","last_peered":"2024-04-24T23:13:27.952365+0000","last_clean":"2024-04-24T23:13:27.952365+0000","last_became_active":"2024-04-24T23:12:26.997661+0000","last_became_peered":"2024-04-24T23:12:26.997661+0000","last_unstale":"2024-04-24T23:13:27.952365+0000","last_undegraded":"2024-04-24T23:13:27.952365+0000","last_fullsized":"2024-04-24T23:13:27.952365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:17.995397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.806718+0000","last_change":"2024-04-24T23:12:28.004822+0000","last_active":"2024-04-24T23:13:56.806718+0000","last_peered":"2024-04-24T23:13:56.806718+0000","last_clean":"2024-04-24T23:13:56.806718+0000","last_became_active":"2024-04-24T23:12:28.004334+0000","last_became_peered":"2024-04-24T23:12:28.004334+0000","last_unstale":"2024-04-24T23:13:56.806718+0000","last_undegraded":"2024-04-24T23:13:56.806718+0000","last_fullsized":"2024-04-24T23:13:56.806718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.717796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952075+0000","last_change":"2024-04-24T23:12:32.085309+0000","last_active":"2024-04-24T23:13:27.952075+0000","last_peered":"2024-04-24T23:13:27.952075+0000","last_clean":"2024-04-24T23:13:27.952075+0000","last_became_active":"2024-04-24T23:12:32.084892+0000","last_became_peered":"2024-04-24T23:12:32.084892+0000","last_unstale":"2024-04-24T23:13:27.952075+0000","last_undegraded":"2024-04-24T23:13:27.952075+0000","last_fullsized":"2024-04-24T23:13:27.952075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:20.318901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953203+0000","last_change":"2024-04-24T23:12:27.005096+0000","last_active":"2024-04-24T23:13:27.953203+0000","last_peered":"2024-04-24T23:13:27.953203+0000","last_clean":"2024-04-24T23:13:27.953203+0000","last_became_active":"2024-04-24T23:12:27.004699+0000","last_became_peered":"2024-04-24T23:12:27.004699+0000","last_unstale":"2024-04-24T23:13:27.953203+0000","last_undegraded":"2024-04-24T23:13:27.953203+0000","last_fullsized":"2024-04-24T23:13:27.953203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:58.225361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.808651+0000","last_change":"2024-04-24T23:12:28.012230+0000","last_active":"2024-04-24T23:13:56.808651+0000","last_peered":"2024-04-24T23:13:56.808651+0000","last_clean":"2024-04-24T23:13:56.808651+0000","last_became_active":"2024-04-24T23:12:28.011900+0000","last_became_peered":"2024-04-24T23:12:28.011900+0000","last_unstale":"2024-04-24T23:13:56.808651+0000","last_undegraded":"2024-04-24T23:13:56.808651+0000","last_fullsized":"2024-04-24T23:13:56.808651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.780548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989677+0000","last_change":"2024-04-24T23:12:32.026388+0000","last_active":"2024-04-24T23:13:27.989677+0000","last_peered":"2024-04-24T23:13:27.989677+0000","last_clean":"2024-04-24T23:13:27.989677+0000","last_became_active":"2024-04-24T23:12:32.025994+0000","last_became_peered":"2024-04-24T23:12:32.025994+0000","last_unstale":"2024-04-24T23:13:27.989677+0000","last_undegraded":"2024-04-24T23:13:27.989677+0000","last_fullsized":"2024-04-24T23:13:27.989677+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:13.062781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:03.975831+0000","last_change":"2024-04-24T23:12:27.005080+0000","last_active":"2024-04-24T23:14:03.975831+0000","last_peered":"2024-04-24T23:14:03.975831+0000","last_clean":"2024-04-24T23:14:03.975831+0000","last_became_active":"2024-04-24T23:12:27.004740+0000","last_became_peered":"2024-04-24T23:12:27.004740+0000","last_unstale":"2024-04-24T23:14:03.975831+0000","last_undegraded":"2024-04-24T23:14:03.975831+0000","last_fullsized":"2024-04-24T23:14:03.975831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:22.502399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.791819+0000","last_change":"2024-04-24T23:12:28.008016+0000","last_active":"2024-04-24T23:13:56.791819+0000","last_peered":"2024-04-24T23:13:56.791819+0000","last_clean":"2024-04-24T23:13:56.791819+0000","last_became_active":"2024-04-24T23:12:28.007139+0000","last_became_peered":"2024-04-24T23:12:28.007139+0000","last_unstale":"2024-04-24T23:13:56.791819+0000","last_undegraded":"2024-04-24T23:13:56.791819+0000","last_fullsized":"2024-04-24T23:13:56.791819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.698193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989740+0000","last_change":"2024-04-24T23:12:32.085290+0000","last_active":"2024-04-24T23:13:27.989740+0000","last_peered":"2024-04-24T23:13:27.989740+0000","last_clean":"2024-04-24T23:13:27.989740+0000","last_became_active":"2024-04-24T23:12:32.084852+0000","last_became_peered":"2024-04-24T23:12:32.084852+0000","last_unstale":"2024-04-24T23:13:27.989740+0000","last_undegraded":"2024-04-24T23:13:27.989740+0000","last_fullsized":"2024-04-24T23:13:27.989740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.753633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.811720+0000","last_change":"2024-04-24T23:12:28.007552+0000","last_active":"2024-04-24T23:13:56.811720+0000","last_peered":"2024-04-24T23:13:56.811720+0000","last_clean":"2024-04-24T23:13:56.811720+0000","last_became_active":"2024-04-24T23:12:28.007196+0000","last_became_peered":"2024-04-24T23:12:28.007196+0000","last_unstale":"2024-04-24T23:13:56.811720+0000","last_undegraded":"2024-04-24T23:13:56.811720+0000","last_fullsized":"2024-04-24T23:13:56.811720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:15.695485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:02.969104+0000","last_change":"2024-04-24T23:12:26.990814+0000","last_active":"2024-04-24T23:14:02.969104+0000","last_peered":"2024-04-24T23:14:02.969104+0000","last_clean":"2024-04-24T23:14:02.969104+0000","last_became_active":"2024-04-24T23:12:26.990476+0000","last_became_peered":"2024-04-24T23:12:26.990476+0000","last_unstale":"2024-04-24T23:14:02.969104+0000","last_undegraded":"2024-04-24T23:14:02.969104+0000","last_fullsized":"2024-04-24T23:14:02.969104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:32.645770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953123+0000","last_change":"2024-04-24T23:12:32.022039+0000","last_active":"2024-04-24T23:13:27.953123+0000","last_peered":"2024-04-24T23:13:27.953123+0000","last_clean":"2024-04-24T23:13:27.953123+0000","last_became_active":"2024-04-24T23:12:32.021514+0000","last_became_peered":"2024-04-24T23:12:32.021514+0000","last_unstale":"2024-04-24T23:13:27.953123+0000","last_undegraded":"2024-04-24T23:13:27.953123+0000","last_fullsized":"2024-04-24T23:13:27.953123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:42.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.804050+0000","last_change":"2024-04-24T23:12:28.008676+0000","last_active":"2024-04-24T23:13:56.804050+0000","last_peered":"2024-04-24T23:13:56.804050+0000","last_clean":"2024-04-24T23:13:56.804050+0000","last_became_active":"2024-04-24T23:12:28.008376+0000","last_became_peered":"2024-04-24T23:12:28.008376+0000","last_unstale":"2024-04-24T23:13:56.804050+0000","last_undegraded":"2024-04-24T23:13:56.804050+0000","last_fullsized":"2024-04-24T23:13:56.804050+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:15.358332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.338508+0000","last_change":"2024-04-24T23:12:26.994952+0000","last_active":"2024-04-24T23:13:28.338508+0000","last_peered":"2024-04-24T23:13:28.338508+0000","last_clean":"2024-04-24T23:13:28.338508+0000","last_became_active":"2024-04-24T23:12:26.994556+0000","last_became_peered":"2024-04-24T23:12:26.994556+0000","last_unstale":"2024-04-24T23:13:28.338508+0000","last_undegraded":"2024-04-24T23:13:28.338508+0000","last_fullsized":"2024-04-24T23:13:28.338508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.257427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953197+0000","last_change":"2024-04-24T23:11:16.721876+0000","last_active":"2024-04-24T23:13:27.953197+0000","last_peered":"2024-04-24T23:13:27.953197+0000","last_clean":"2024-04-24T23:13:27.953197+0000","last_became_active":"2024-04-24T23:11:16.721291+0000","last_became_peered":"2024-04-24T23:11:16.721291+0000","last_unstale":"2024-04-24T23:13:27.953197+0000","last_undegraded":"2024-04-24T23:13:27.953197+0000","last_fullsized":"2024-04-24T23:13:27.953197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_clean_scrub_stamp":"2024-04-24T23:07:51.257753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.458939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955980+0000","last_change":"2024-04-24T23:12:32.085905+0000","last_active":"2024-04-24T23:13:27.955980+0000","last_peered":"2024-04-24T23:13:27.955980+0000","last_clean":"2024-04-24T23:13:27.955980+0000","last_became_active":"2024-04-24T23:12:32.085374+0000","last_became_peered":"2024-04-24T23:12:32.085374+0000","last_unstale":"2024-04-24T23:13:27.955980+0000","last_undegraded":"2024-04-24T23:13:27.955980+0000","last_fullsized":"2024-04-24T23:13:27.955980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:16.290651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.806581+0000","last_change":"2024-04-24T23:12:27.999880+0000","last_active":"2024-04-24T23:13:56.806581+0000","last_peered":"2024-04-24T23:13:56.806581+0000","last_clean":"2024-04-24T23:13:56.806581+0000","last_became_active":"2024-04-24T23:12:27.999543+0000","last_became_peered":"2024-04-24T23:12:27.999543+0000","last_unstale":"2024-04-24T23:13:56.806581+0000","last_undegraded":"2024-04-24T23:13:56.806581+0000","last_fullsized":"2024-04-24T23:13:56.806581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:44.758968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.341009+0000","last_change":"2024-04-24T23:13:28.341009+0000","last_active":"2024-04-24T23:13:28.341009+0000","last_peered":"2024-04-24T23:13:28.341009+0000","last_clean":"2024-04-24T23:13:28.341009+0000","last_became_active":"2024-04-24T23:13:28.338192+0000","last_became_peered":"2024-04-24T23:13:28.338192+0000","last_unstale":"2024-04-24T23:13:28.341009+0000","last_undegraded":"2024-04-24T23:13:28.341009+0000","last_fullsized":"2024-04-24T23:13:28.341009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:11.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.954033+0000","last_change":"2024-04-24T23:12:32.024952+0000","last_active":"2024-04-24T23:13:27.954033+0000","last_peered":"2024-04-24T23:13:27.954033+0000","last_clean":"2024-04-24T23:13:27.954033+0000","last_became_active":"2024-04-24T23:12:32.024404+0000","last_became_peered":"2024-04-24T23:12:32.024404+0000","last_unstale":"2024-04-24T23:13:27.954033+0000","last_undegraded":"2024-04-24T23:13:27.954033+0000","last_fullsized":"2024-04-24T23:13:27.954033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.377794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.952697+0000","last_change":"2024-04-24T23:12:27.007191+0000","last_active":"2024-04-24T23:13:27.952697+0000","last_peered":"2024-04-24T23:13:27.952697+0000","last_clean":"2024-04-24T23:13:27.952697+0000","last_became_active":"2024-04-24T23:12:27.006867+0000","last_became_peered":"2024-04-24T23:12:27.006867+0000","last_unstale":"2024-04-24T23:13:27.952697+0000","last_undegraded":"2024-04-24T23:13:27.952697+0000","last_fullsized":"2024-04-24T23:13:27.952697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:58.112293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.809060+0000","last_change":"2024-04-24T23:12:27.996322+0000","last_active":"2024-04-24T23:13:56.809060+0000","last_peered":"2024-04-24T23:13:56.809060+0000","last_clean":"2024-04-24T23:13:56.809060+0000","last_became_active":"2024-04-24T23:12:27.995991+0000","last_became_peered":"2024-04-24T23:12:27.995991+0000","last_unstale":"2024-04-24T23:13:56.809060+0000","last_undegraded":"2024-04-24T23:13:56.809060+0000","last_fullsized":"2024-04-24T23:13:56.809060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:08.960347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956492+0000","last_change":"2024-04-24T23:12:32.084959+0000","last_active":"2024-04-24T23:13:27.956492+0000","last_peered":"2024-04-24T23:13:27.956492+0000","last_clean":"2024-04-24T23:13:27.956492+0000","last_became_active":"2024-04-24T23:12:32.084553+0000","last_became_peered":"2024-04-24T23:12:32.084553+0000","last_unstale":"2024-04-24T23:13:27.956492+0000","last_undegraded":"2024-04-24T23:13:27.956492+0000","last_fullsized":"2024-04-24T23:13:27.956492+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.836337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953483+0000","last_change":"2024-04-24T23:12:27.007742+0000","last_active":"2024-04-24T23:13:27.953483+0000","last_peered":"2024-04-24T23:13:27.953483+0000","last_clean":"2024-04-24T23:13:27.953483+0000","last_became_active":"2024-04-24T23:12:27.007423+0000","last_became_peered":"2024-04-24T23:12:27.007423+0000","last_unstale":"2024-04-24T23:13:27.953483+0000","last_undegraded":"2024-04-24T23:13:27.953483+0000","last_fullsized":"2024-04-24T23:13:27.953483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:39.478356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.804047+0000","last_change":"2024-04-24T23:12:28.010961+0000","last_active":"2024-04-24T23:13:56.804047+0000","last_peered":"2024-04-24T23:13:56.804047+0000","last_clean":"2024-04-24T23:13:56.804047+0000","last_became_active":"2024-04-24T23:12:28.010641+0000","last_became_peered":"2024-04-24T23:12:28.010641+0000","last_unstale":"2024-04-24T23:13:56.804047+0000","last_undegraded":"2024-04-24T23:13:56.804047+0000","last_fullsized":"2024-04-24T23:13:56.804047+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:43.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956688+0000","last_change":"2024-04-24T23:12:32.023687+0000","last_active":"2024-04-24T23:13:27.956688+0000","last_peered":"2024-04-24T23:13:27.956688+0000","last_clean":"2024-04-24T23:13:27.956688+0000","last_became_active":"2024-04-24T23:12:32.023052+0000","last_became_peered":"2024-04-24T23:12:32.023052+0000","last_unstale":"2024-04-24T23:13:27.956688+0000","last_undegraded":"2024-04-24T23:13:27.956688+0000","last_fullsized":"2024-04-24T23:13:27.956688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:39.854030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.337090+0000","last_change":"2024-04-24T23:12:27.009743+0000","last_active":"2024-04-24T23:13:28.337090+0000","last_peered":"2024-04-24T23:13:28.337090+0000","last_clean":"2024-04-24T23:13:28.337090+0000","last_became_active":"2024-04-24T23:12:27.009343+0000","last_became_peered":"2024-04-24T23:12:27.009343+0000","last_unstale":"2024-04-24T23:13:28.337090+0000","last_undegraded":"2024-04-24T23:13:28.337090+0000","last_fullsized":"2024-04-24T23:13:28.337090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:20.387738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.818082+0000","last_change":"2024-04-24T23:13:27.961769+0000","last_active":"2024-04-24T23:13:56.818082+0000","last_peered":"2024-04-24T23:13:56.818082+0000","last_clean":"2024-04-24T23:13:56.818082+0000","last_became_active":"2024-04-24T23:13:27.961460+0000","last_became_peered":"2024-04-24T23:13:27.961460+0000","last_unstale":"2024-04-24T23:13:56.818082+0000","last_undegraded":"2024-04-24T23:13:56.818082+0000","last_fullsized":"2024-04-24T23:13:56.818082+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:01.012688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336604+0000","last_change":"2024-04-24T23:12:32.086209+0000","last_active":"2024-04-24T23:13:28.336604+0000","last_peered":"2024-04-24T23:13:28.336604+0000","last_clean":"2024-04-24T23:13:28.336604+0000","last_became_active":"2024-04-24T23:12:32.085752+0000","last_became_peered":"2024-04-24T23:12:32.085752+0000","last_unstale":"2024-04-24T23:13:28.336604+0000","last_undegraded":"2024-04-24T23:13:28.336604+0000","last_fullsized":"2024-04-24T23:13:28.336604+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:52.120905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.337370+0000","last_change":"2024-04-24T23:12:26.996355+0000","last_active":"2024-04-24T23:13:28.337370+0000","last_peered":"2024-04-24T23:13:28.337370+0000","last_clean":"2024-04-24T23:13:28.337370+0000","last_became_active":"2024-04-24T23:12:26.995486+0000","last_became_peered":"2024-04-24T23:12:26.995486+0000","last_unstale":"2024-04-24T23:13:28.337370+0000","last_undegraded":"2024-04-24T23:13:28.337370+0000","last_fullsized":"2024-04-24T23:13:28.337370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:48.981985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.794229+0000","last_change":"2024-04-24T23:12:28.004610+0000","last_active":"2024-04-24T23:13:56.794229+0000","last_peered":"2024-04-24T23:13:56.794229+0000","last_clean":"2024-04-24T23:13:56.794229+0000","last_became_active":"2024-04-24T23:12:28.004051+0000","last_became_peered":"2024-04-24T23:12:28.004051+0000","last_unstale":"2024-04-24T23:13:56.794229+0000","last_undegraded":"2024-04-24T23:13:56.794229+0000","last_fullsized":"2024-04-24T23:13:56.794229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:42.656954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.953447+0000","last_change":"2024-04-24T23:12:32.029269+0000","last_active":"2024-04-24T23:13:27.953447+0000","last_peered":"2024-04-24T23:13:27.953447+0000","last_clean":"2024-04-24T23:13:27.953447+0000","last_became_active":"2024-04-24T23:12:32.028880+0000","last_became_peered":"2024-04-24T23:12:32.028880+0000","last_unstale":"2024-04-24T23:13:27.953447+0000","last_undegraded":"2024-04-24T23:13:27.953447+0000","last_fullsized":"2024-04-24T23:13:27.953447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:19.806267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956400+0000","last_change":"2024-04-24T23:12:27.006357+0000","last_active":"2024-04-24T23:13:27.956400+0000","last_peered":"2024-04-24T23:13:27.956400+0000","last_clean":"2024-04-24T23:13:27.956400+0000","last_became_active":"2024-04-24T23:12:27.006040+0000","last_became_peered":"2024-04-24T23:12:27.006040+0000","last_unstale":"2024-04-24T23:13:27.956400+0000","last_undegraded":"2024-04-24T23:13:27.956400+0000","last_fullsized":"2024-04-24T23:13:27.956400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:27.520663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.804128+0000","last_change":"2024-04-24T23:12:28.008909+0000","last_active":"2024-04-24T23:13:56.804128+0000","last_peered":"2024-04-24T23:13:56.804128+0000","last_clean":"2024-04-24T23:13:56.804128+0000","last_became_active":"2024-04-24T23:12:28.008552+0000","last_became_peered":"2024-04-24T23:12:28.008552+0000","last_unstale":"2024-04-24T23:13:56.804128+0000","last_undegraded":"2024-04-24T23:13:56.804128+0000","last_fullsized":"2024-04-24T23:13:56.804128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:20.628234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336135+0000","last_change":"2024-04-24T23:12:32.029720+0000","last_active":"2024-04-24T23:13:28.336135+0000","last_peered":"2024-04-24T23:13:28.336135+0000","last_clean":"2024-04-24T23:13:28.336135+0000","last_became_active":"2024-04-24T23:12:32.029311+0000","last_became_peered":"2024-04-24T23:12:32.029311+0000","last_unstale":"2024-04-24T23:13:28.336135+0000","last_undegraded":"2024-04-24T23:13:28.336135+0000","last_fullsized":"2024-04-24T23:13:28.336135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:37.832102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.956975+0000","last_change":"2024-04-24T23:12:26.995552+0000","last_active":"2024-04-24T23:13:27.956975+0000","last_peered":"2024-04-24T23:13:27.956975+0000","last_clean":"2024-04-24T23:13:27.956975+0000","last_became_active":"2024-04-24T23:12:26.994922+0000","last_became_peered":"2024-04-24T23:12:26.994922+0000","last_unstale":"2024-04-24T23:13:27.956975+0000","last_undegraded":"2024-04-24T23:13:27.956975+0000","last_fullsized":"2024-04-24T23:13:27.956975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:17.981344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.803824+0000","last_change":"2024-04-24T23:12:28.008802+0000","last_active":"2024-04-24T23:13:56.803824+0000","last_peered":"2024-04-24T23:13:56.803824+0000","last_clean":"2024-04-24T23:13:56.803824+0000","last_became_active":"2024-04-24T23:12:28.008496+0000","last_became_peered":"2024-04-24T23:12:28.008496+0000","last_unstale":"2024-04-24T23:13:56.803824+0000","last_undegraded":"2024-04-24T23:13:56.803824+0000","last_fullsized":"2024-04-24T23:13:56.803824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:54.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336714+0000","last_change":"2024-04-24T23:12:32.085555+0000","last_active":"2024-04-24T23:13:28.336714+0000","last_peered":"2024-04-24T23:13:28.336714+0000","last_clean":"2024-04-24T23:13:28.336714+0000","last_became_active":"2024-04-24T23:12:32.085085+0000","last_became_peered":"2024-04-24T23:12:32.085085+0000","last_unstale":"2024-04-24T23:13:28.336714+0000","last_undegraded":"2024-04-24T23:13:28.336714+0000","last_fullsized":"2024-04-24T23:13:28.336714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:24.563964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.342962+0000","last_change":"2024-04-24T23:13:28.342962+0000","last_active":"2024-04-24T23:13:28.342962+0000","last_peered":"2024-04-24T23:13:28.342962+0000","last_clean":"2024-04-24T23:13:28.342962+0000","last_became_active":"2024-04-24T23:13:28.339742+0000","last_became_peered":"2024-04-24T23:13:28.339742+0000","last_unstale":"2024-04-24T23:13:28.342962+0000","last_undegraded":"2024-04-24T23:13:28.342962+0000","last_fullsized":"2024-04-24T23:13:28.342962+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:36.910863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.820298+0000","last_change":"2024-04-24T23:12:28.014865+0000","last_active":"2024-04-24T23:13:56.820298+0000","last_peered":"2024-04-24T23:13:56.820298+0000","last_clean":"2024-04-24T23:13:56.820298+0000","last_became_active":"2024-04-24T23:12:28.014538+0000","last_became_peered":"2024-04-24T23:12:28.014538+0000","last_unstale":"2024-04-24T23:13:56.820298+0000","last_undegraded":"2024-04-24T23:13:56.820298+0000","last_fullsized":"2024-04-24T23:13:56.820298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:55.306883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:28.336135+0000","last_change":"2024-04-24T23:12:32.024849+0000","last_active":"2024-04-24T23:13:28.336135+0000","last_peered":"2024-04-24T23:13:28.336135+0000","last_clean":"2024-04-24T23:13:28.336135+0000","last_became_active":"2024-04-24T23:12:32.024412+0000","last_became_peered":"2024-04-24T23:12:32.024412+0000","last_unstale":"2024-04-24T23:13:28.336135+0000","last_undegraded":"2024-04-24T23:13:28.336135+0000","last_fullsized":"2024-04-24T23:13:28.336135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:16.884565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":104,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:14:03.976367+0000","last_change":"2024-04-24T23:12:27.007284+0000","last_active":"2024-04-24T23:14:03.976367+0000","last_peered":"2024-04-24T23:14:03.976367+0000","last_clean":"2024-04-24T23:14:03.976367+0000","last_became_active":"2024-04-24T23:12:27.006916+0000","last_became_peered":"2024-04-24T23:12:27.006916+0000","last_unstale":"2024-04-24T23:14:03.976367+0000","last_undegraded":"2024-04-24T23:14:03.976367+0000","last_fullsized":"2024-04-24T23:14:03.976367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:40.333387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821395+0000","last_change":"2024-04-24T23:12:28.011603+0000","last_active":"2024-04-24T23:13:56.821395+0000","last_peered":"2024-04-24T23:13:56.821395+0000","last_clean":"2024-04-24T23:13:56.821395+0000","last_became_active":"2024-04-24T23:12:28.011268+0000","last_became_peered":"2024-04-24T23:12:28.011268+0000","last_unstale":"2024-04-24T23:13:56.821395+0000","last_undegraded":"2024-04-24T23:13:56.821395+0000","last_fullsized":"2024-04-24T23:13:56.821395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.693024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955139+0000","last_change":"2024-04-24T23:12:32.033965+0000","last_active":"2024-04-24T23:13:27.955139+0000","last_peered":"2024-04-24T23:13:27.955139+0000","last_clean":"2024-04-24T23:13:27.955139+0000","last_became_active":"2024-04-24T23:12:32.033472+0000","last_became_peered":"2024-04-24T23:12:32.033472+0000","last_unstale":"2024-04-24T23:13:27.955139+0000","last_undegraded":"2024-04-24T23:13:27.955139+0000","last_fullsized":"2024-04-24T23:13:27.955139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:05.243047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.957238+0000","last_change":"2024-04-24T23:12:26.991981+0000","last_active":"2024-04-24T23:13:27.957238+0000","last_peered":"2024-04-24T23:13:27.957238+0000","last_clean":"2024-04-24T23:13:27.957238+0000","last_became_active":"2024-04-24T23:12:26.990707+0000","last_became_peered":"2024-04-24T23:12:26.990707+0000","last_unstale":"2024-04-24T23:13:27.957238+0000","last_undegraded":"2024-04-24T23:13:27.957238+0000","last_fullsized":"2024-04-24T23:13:27.957238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:36.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.817461+0000","last_change":"2024-04-24T23:12:28.010428+0000","last_active":"2024-04-24T23:13:56.817461+0000","last_peered":"2024-04-24T23:13:56.817461+0000","last_clean":"2024-04-24T23:13:56.817461+0000","last_became_active":"2024-04-24T23:12:28.010096+0000","last_became_peered":"2024-04-24T23:12:28.010096+0000","last_unstale":"2024-04-24T23:13:56.817461+0000","last_undegraded":"2024-04-24T23:13:56.817461+0000","last_fullsized":"2024-04-24T23:13:56.817461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:31.949617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.955329+0000","last_change":"2024-04-24T23:12:32.023828+0000","last_active":"2024-04-24T23:13:27.955329+0000","last_peered":"2024-04-24T23:13:27.955329+0000","last_clean":"2024-04-24T23:13:27.955329+0000","last_became_active":"2024-04-24T23:12:32.023437+0000","last_became_peered":"2024-04-24T23:12:32.023437+0000","last_unstale":"2024-04-24T23:13:27.955329+0000","last_undegraded":"2024-04-24T23:13:27.955329+0000","last_fullsized":"2024-04-24T23:13:27.955329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.186860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:27.989812+0000","last_change":"2024-04-24T23:12:26.993711+0000","last_active":"2024-04-24T23:13:27.989812+0000","last_peered":"2024-04-24T23:13:27.989812+0000","last_clean":"2024-04-24T23:13:27.989812+0000","last_became_active":"2024-04-24T23:12:26.993368+0000","last_became_peered":"2024-04-24T23:12:26.993368+0000","last_unstale":"2024-04-24T23:13:27.989812+0000","last_undegraded":"2024-04-24T23:13:27.989812+0000","last_fullsized":"2024-04-24T23:13:27.989812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:58.843747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.811933+0000","last_change":"2024-04-24T23:12:28.003026+0000","last_active":"2024-04-24T23:13:56.811933+0000","last_peered":"2024-04-24T23:13:56.811933+0000","last_clean":"2024-04-24T23:13:56.811933+0000","last_became_active":"2024-04-24T23:12:28.002693+0000","last_became_peered":"2024-04-24T23:12:28.002693+0000","last_unstale":"2024-04-24T23:13:56.811933+0000","last_undegraded":"2024-04-24T23:13:56.811933+0000","last_fullsized":"2024-04-24T23:13:56.811933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:30.926251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.807841+0000","last_change":"2024-04-24T23:12:28.013414+0000","last_active":"2024-04-24T23:13:56.807841+0000","last_peered":"2024-04-24T23:13:56.807841+0000","last_clean":"2024-04-24T23:13:56.807841+0000","last_became_active":"2024-04-24T23:12:28.013030+0000","last_became_peered":"2024-04-24T23:12:28.013030+0000","last_unstale":"2024-04-24T23:13:56.807841+0000","last_undegraded":"2024-04-24T23:13:56.807841+0000","last_fullsized":"2024-04-24T23:13:56.807841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:48.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.809798+0000","last_change":"2024-04-24T23:12:28.004821+0000","last_active":"2024-04-24T23:13:56.809798+0000","last_peered":"2024-04-24T23:13:56.809798+0000","last_clean":"2024-04-24T23:13:56.809798+0000","last_became_active":"2024-04-24T23:12:28.003162+0000","last_became_peered":"2024-04-24T23:12:28.003162+0000","last_unstale":"2024-04-24T23:13:56.809798+0000","last_undegraded":"2024-04-24T23:13:56.809798+0000","last_fullsized":"2024-04-24T23:13:56.809798+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:59.109665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.809391+0000","last_change":"2024-04-24T23:12:28.002115+0000","last_active":"2024-04-24T23:13:56.809391+0000","last_peered":"2024-04-24T23:13:56.809391+0000","last_clean":"2024-04-24T23:13:56.809391+0000","last_became_active":"2024-04-24T23:12:28.001727+0000","last_became_peered":"2024-04-24T23:12:28.001727+0000","last_unstale":"2024-04-24T23:13:56.809391+0000","last_undegraded":"2024-04-24T23:13:56.809391+0000","last_fullsized":"2024-04-24T23:13:56.809391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:14.348843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.812005+0000","last_change":"2024-04-24T23:12:28.007784+0000","last_active":"2024-04-24T23:13:56.812005+0000","last_peered":"2024-04-24T23:13:56.812005+0000","last_clean":"2024-04-24T23:13:56.812005+0000","last_became_active":"2024-04-24T23:12:28.007481+0000","last_became_peered":"2024-04-24T23:12:28.007481+0000","last_unstale":"2024-04-24T23:13:56.812005+0000","last_undegraded":"2024-04-24T23:13:56.812005+0000","last_fullsized":"2024-04-24T23:13:56.812005+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:33.753448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.803491+0000","last_change":"2024-04-24T23:12:28.009100+0000","last_active":"2024-04-24T23:13:56.803491+0000","last_peered":"2024-04-24T23:13:56.803491+0000","last_clean":"2024-04-24T23:13:56.803491+0000","last_became_active":"2024-04-24T23:12:28.008782+0000","last_became_peered":"2024-04-24T23:12:28.008782+0000","last_unstale":"2024-04-24T23:13:56.803491+0000","last_undegraded":"2024-04-24T23:13:56.803491+0000","last_fullsized":"2024-04-24T23:13:56.803491+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:44.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.805972+0000","last_change":"2024-04-24T23:12:28.008129+0000","last_active":"2024-04-24T23:13:56.805972+0000","last_peered":"2024-04-24T23:13:56.805972+0000","last_clean":"2024-04-24T23:13:56.805972+0000","last_became_active":"2024-04-24T23:12:28.007819+0000","last_became_peered":"2024-04-24T23:12:28.007819+0000","last_unstale":"2024-04-24T23:13:56.805972+0000","last_undegraded":"2024-04-24T23:13:56.805972+0000","last_fullsized":"2024-04-24T23:13:56.805972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.234705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.805616+0000","last_change":"2024-04-24T23:12:27.993188+0000","last_active":"2024-04-24T23:13:56.805616+0000","last_peered":"2024-04-24T23:13:56.805616+0000","last_clean":"2024-04-24T23:13:56.805616+0000","last_became_active":"2024-04-24T23:12:27.992832+0000","last_became_peered":"2024-04-24T23:12:27.992832+0000","last_unstale":"2024-04-24T23:13:56.805616+0000","last_undegraded":"2024-04-24T23:13:56.805616+0000","last_fullsized":"2024-04-24T23:13:56.805616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:58.623164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.807479+0000","last_change":"2024-04-24T23:12:28.004099+0000","last_active":"2024-04-24T23:13:56.807479+0000","last_peered":"2024-04-24T23:13:56.807479+0000","last_clean":"2024-04-24T23:13:56.807479+0000","last_became_active":"2024-04-24T23:12:28.003802+0000","last_became_peered":"2024-04-24T23:12:28.003802+0000","last_unstale":"2024-04-24T23:13:56.807479+0000","last_undegraded":"2024-04-24T23:13:56.807479+0000","last_fullsized":"2024-04-24T23:13:56.807479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:25.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.806152+0000","last_change":"2024-04-24T23:12:28.007700+0000","last_active":"2024-04-24T23:13:56.806152+0000","last_peered":"2024-04-24T23:13:56.806152+0000","last_clean":"2024-04-24T23:13:56.806152+0000","last_became_active":"2024-04-24T23:12:28.002923+0000","last_became_peered":"2024-04-24T23:12:28.002923+0000","last_unstale":"2024-04-24T23:13:56.806152+0000","last_undegraded":"2024-04-24T23:13:56.806152+0000","last_fullsized":"2024-04-24T23:13:56.806152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:46.736096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.810863+0000","last_change":"2024-04-24T23:12:28.002194+0000","last_active":"2024-04-24T23:13:56.810863+0000","last_peered":"2024-04-24T23:13:56.810863+0000","last_clean":"2024-04-24T23:13:56.810863+0000","last_became_active":"2024-04-24T23:12:28.001703+0000","last_became_peered":"2024-04-24T23:12:28.001703+0000","last_unstale":"2024-04-24T23:13:56.810863+0000","last_undegraded":"2024-04-24T23:13:56.810863+0000","last_fullsized":"2024-04-24T23:13:56.810863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:20.887025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.821672+0000","last_change":"2024-04-24T23:12:28.015980+0000","last_active":"2024-04-24T23:13:56.821672+0000","last_peered":"2024-04-24T23:13:56.821672+0000","last_clean":"2024-04-24T23:13:56.821672+0000","last_became_active":"2024-04-24T23:12:28.013510+0000","last_became_peered":"2024-04-24T23:12:28.013510+0000","last_unstale":"2024-04-24T23:13:56.821672+0000","last_undegraded":"2024-04-24T23:13:56.821672+0000","last_fullsized":"2024-04-24T23:13:56.821672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:37.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.816253+0000","last_change":"2024-04-24T23:12:28.001883+0000","last_active":"2024-04-24T23:13:56.816253+0000","last_peered":"2024-04-24T23:13:56.816253+0000","last_clean":"2024-04-24T23:13:56.816253+0000","last_became_active":"2024-04-24T23:12:28.001499+0000","last_became_peered":"2024-04-24T23:12:28.001499+0000","last_unstale":"2024-04-24T23:13:56.816253+0000","last_undegraded":"2024-04-24T23:13:56.816253+0000","last_fullsized":"2024-04-24T23:13:56.816253+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:46.805548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-24T23:13:56.803957+0000","last_change":"2024-04-24T23:12:28.004660+0000","last_active":"2024-04-24T23:13:56.803957+0000","last_peered":"2024-04-24T23:13:56.803957+0000","last_clean":"2024-04-24T23:13:56.803957+0000","last_became_active":"2024-04-24T23:12:28.004328+0000","last_became_peered":"2024-04-24T23:12:28.004328+0000","last_unstale":"2024-04-24T23:13:56.803957+0000","last_undegraded":"2024-04-24T23:13:56.803957+0000","last_fullsized":"2024-04-24T23:13:56.803957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:04.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70559,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":765,"num_read_kb":464,"num_write":248,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":506099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612647,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32876,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712276,"statfs":{"total":95995035648,"available":95961370624,"internally_reserved":0,"allocated":1740800,"data_stored":1156340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1073152,"data_stored":509945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":506115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28640,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716512,"statfs":{"total":95995035648,"available":95965708288,"internally_reserved":0,"allocated":1662976,"data_stored":1102865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":501353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":565988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":502809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":566672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":501061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":505427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36968,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708184,"statfs":{"total":95995035648,"available":95957180416,"internally_reserved":0,"allocated":1671168,"data_stored":1098447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:14:09.914 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T23:14:09.914 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T23:14:09.915 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T23:14:09.915 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T23:14:10.120 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:09 smithi150 ceph-mon[32247]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:09 smithi187 ceph-mon[33292]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:10.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:09 smithi112 ceph-mon[29186]: pgmap v343: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:11.130 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:10 smithi112 ceph-mon[29186]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:11.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:10 smithi187 ceph-mon[33292]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:11.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:10 smithi150 ceph-mon[32247]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 -- 172.21.15.112:0/1683676247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb410a4a0 msgr2=0x7f7cb410a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 --2- 172.21.15.112:0/1683676247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb410a4a0 0x7f7cb410a880 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009990 tx=0x7f7cb0009ca0 comp rx=0 tx=0).stop 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 -- 172.21.15.112:0/1683676247 shutdown_connections 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 --2- 172.21.15.112:0/1683676247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb4104dd0 0x7f7cb41112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 --2- 172.21.15.112:0/1683676247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb410a4a0 0x7f7cb410a880 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 --2- 172.21.15.112:0/1683676247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cb4104410 0x7f7cb4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 -- 172.21.15.112:0/1683676247 >> 172.21.15.112:0/1683676247 conn(0x7f7cb40ffee0 msgr2=0x7f7cb4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 -- 172.21.15.112:0/1683676247 shutdown_connections 2024-04-24T23:14:11.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.894+0000 7f7cbc315700 1 -- 172.21.15.112:0/1683676247 wait complete. 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cbc315700 1 Processor -- start 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cbc315700 1 -- start start 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cbc315700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cb4104410 0x7f7cb41a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cbc315700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb4104dd0 0x7f7cb41a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cbc315700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb410a4a0 0x7f7cb419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cbc315700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4114880 con 0x7f7cb4104dd0 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cbc315700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4114700 con 0x7f7cb410a4a0 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cbc315700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4114a00 con 0x7f7cb4104410 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb4104dd0 0x7f7cb41a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb4104dd0 0x7f7cb41a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56090/0 (socket says 172.21.15.112:56090) 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 -- 172.21.15.112:0/3238587526 learned_addr learned my addr 172.21.15.112:0/3238587526 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cba0b1700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cb4104410 0x7f7cb41a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:11.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cba8b2700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb410a4a0 0x7f7cb419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 -- 172.21.15.112:0/3238587526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cb4104410 msgr2=0x7f7cb41a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cb4104410 0x7f7cb41a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 -- 172.21.15.112:0/3238587526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb410a4a0 msgr2=0x7f7cb419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb410a4a0 0x7f7cb419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 -- 172.21.15.112:0/3238587526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb419b280 con 0x7f7cb4104dd0 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.895+0000 7f7cb98b0700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb4104dd0 0x7f7cb41a0e50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c970 tx=0x7f7ca400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.896+0000 7f7cab7fe700 1 -- 172.21.15.112:0/3238587526 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca40078d0 con 0x7f7cb4104dd0 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.896+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb419b570 con 0x7f7cb4104dd0 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.896+0000 7f7cab7fe700 1 -- 172.21.15.112:0/3238587526 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7ca4018480 con 0x7f7cb4104dd0 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.896+0000 7f7cab7fe700 1 -- 172.21.15.112:0/3238587526 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400f6b0 con 0x7f7cb4104dd0 2024-04-24T23:14:11.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.896+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb41a77e0 con 0x7f7cb4104dd0 2024-04-24T23:14:11.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.897+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb410c9a0 con 0x7f7cb4104dd0 2024-04-24T23:14:11.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.898+0000 7f7cab7fe700 1 -- 172.21.15.112:0/3238587526 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7ca4018630 con 0x7f7cb4104dd0 2024-04-24T23:14:11.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.898+0000 7f7cab7fe700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7ca00a2c20 0x7f7ca00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:11.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.898+0000 7f7cba0b1700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7ca00a2c20 0x7f7ca00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:11.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.898+0000 7f7cab7fe700 1 -- 172.21.15.112:0/3238587526 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7ca40cf4b0 con 0x7f7cb4104dd0 2024-04-24T23:14:11.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.899+0000 7f7cba0b1700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7ca00a2c20 0x7f7ca00a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000b5c0 tx=0x7f7cb0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:11.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:11.903+0000 7f7cab7fe700 1 -- 172.21.15.112:0/3238587526 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ca409c140 con 0x7f7cb4104dd0 2024-04-24T23:14:12.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.150+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7cb419bfc0 con 0x7f7cb4104dd0 2024-04-24T23:14:12.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:11 smithi112 ceph-mon[29186]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-24T23:14:12.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.151+0000 7f7cab7fe700 1 -- 172.21.15.112:0/3238587526 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7ca409ba40 con 0x7f7cb4104dd0 2024-04-24T23:14:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:12.154 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7ca00a2c20 msgr2=0x7f7ca00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7ca00a2c20 0x7f7ca00a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7cb000b5c0 tx=0x7f7cb0005960 comp rx=0 tx=0).stop 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb4104dd0 msgr2=0x7f7cb41a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb4104dd0 0x7f7cb41a0e50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c970 tx=0x7f7ca400cc80 comp rx=0 tx=0).stop 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 shutdown_connections 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7ca00a2c20 0x7f7ca00a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb410a4a0 0x7f7cb419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cb4104410 0x7f7cb41a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 --2- 172.21.15.112:0/3238587526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cb4104dd0 0x7f7cb41a0e50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 >> 172.21.15.112:0/3238587526 conn(0x7f7cb40ffee0 msgr2=0x7f7cb41004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:12.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 shutdown_connections 2024-04-24T23:14:12.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.154+0000 7f7cbc315700 1 -- 172.21.15.112:0/3238587526 wait complete. 2024-04-24T23:14:12.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:11 smithi187 ceph-mon[33292]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-24T23:14:12.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:11 smithi150 ceph-mon[32247]: pgmap v344: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-24T23:14:12.852 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T23:14:12.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 -- 172.21.15.112:0/4040676064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 msgr2=0x7f9a24100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/4040676064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 0x7f9a24100aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009900 tx=0x7f9a1402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 -- 172.21.15.112:0/4040676064 shutdown_connections 2024-04-24T23:14:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/4040676064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a24102230 0x7f9a24108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/4040676064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a24101870 0x7f9a24101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/4040676064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 0x7f9a24100aa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 -- 172.21.15.112:0/4040676064 >> 172.21.15.112:0/4040676064 conn(0x7f9a240fbbf0 msgr2=0x7f9a240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 -- 172.21.15.112:0/4040676064 shutdown_connections 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.953+0000 7f9a28ba2640 1 -- 172.21.15.112:0/4040676064 wait complete. 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a28ba2640 1 Processor -- start 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a28ba2640 1 -- start start 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a28ba2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 0x7f9a24069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a28ba2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a24101870 0x7f9a2419c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a28ba2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a24102230 0x7f9a2419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a28ba2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2410c430 con 0x7f9a24100680 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a28ba2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2410c2b0 con 0x7f9a24102230 2024-04-24T23:14:12.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a28ba2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2410c5b0 con 0x7f9a24101870 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a2259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 0x7f9a24069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a2259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 0x7f9a24069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56114/0 (socket says 172.21.15.112:56114) 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a2259c640 1 -- 172.21.15.112:0/3300266904 learned_addr learned my addr 172.21.15.112:0/3300266904 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a1bfff640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a24101870 0x7f9a2419c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a22d9d640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a24102230 0x7f9a2419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a2259c640 1 -- 172.21.15.112:0/3300266904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a24101870 msgr2=0x7f9a2419c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a2259c640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a24101870 0x7f9a2419c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a2259c640 1 -- 172.21.15.112:0/3300266904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a24102230 msgr2=0x7f9a2419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a2259c640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a24102230 0x7f9a2419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.954+0000 7f9a2259c640 1 -- 172.21.15.112:0/3300266904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2419f780 con 0x7f9a24100680 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.955+0000 7f9a2259c640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 0x7f9a24069460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009230 tx=0x7f9a140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:12.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.955+0000 7f9a1b7fe640 1 -- 172.21.15.112:0/3300266904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1403f030 con 0x7f9a24100680 2024-04-24T23:14:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.955+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2419fa10 con 0x7f9a24100680 2024-04-24T23:14:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.955+0000 7f9a1b7fe640 1 -- 172.21.15.112:0/3300266904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9a14003d00 con 0x7f9a24100680 2024-04-24T23:14:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.955+0000 7f9a1b7fe640 1 -- 172.21.15.112:0/3300266904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14035070 con 0x7f9a24100680 2024-04-24T23:14:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.955+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2419ff00 con 0x7f9a24100680 2024-04-24T23:14:12.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.956+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a241079a0 con 0x7f9a24100680 2024-04-24T23:14:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.957+0000 7f9a1b7fe640 1 -- 172.21.15.112:0/3300266904 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9a14042030 con 0x7f9a24100680 2024-04-24T23:14:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.957+0000 7f9a1b7fe640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9a100a2b90 0x7f9a100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.958+0000 7f9a1bfff640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9a100a2b90 0x7f9a100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.958+0000 7f9a1b7fe640 1 -- 172.21.15.112:0/3300266904 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9a140efd20 con 0x7f9a24100680 2024-04-24T23:14:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.958+0000 7f9a1bfff640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9a100a2b90 0x7f9a100a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a240691d0 tx=0x7f9a0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:12.961+0000 7f9a1b7fe640 1 -- 172.21.15.112:0/3300266904 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9a140bc920 con 0x7f9a24100680 2024-04-24T23:14:13.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.138+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a2402d3c0 con 0x7f9a24100680 2024-04-24T23:14:13.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.139+0000 7f9a1b7fe640 1 -- 172.21.15.112:0/3300266904 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24714 (secure 0 0 0) 0x7f9a140bc220 con 0x7f9a24100680 2024-04-24T23:14:13.141 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:13.141 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":92,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:13:27.938988+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:13.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.141+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9a100a2b90 msgr2=0x7f9a100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.141+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9a100a2b90 0x7f9a100a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9a240691d0 tx=0x7f9a0c005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.141+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 msgr2=0x7f9a24069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.141+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 0x7f9a24069460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009230 tx=0x7f9a140042f0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.142+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 shutdown_connections 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.142+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9a100a2b90 0x7f9a100a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.142+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a24102230 0x7f9a2419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.142+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a24101870 0x7f9a2419c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.142+0000 7f9a28ba2640 1 --2- 172.21.15.112:0/3300266904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a24100680 0x7f9a24069460 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.142+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 >> 172.21.15.112:0/3300266904 conn(0x7f9a240fbbf0 msgr2=0x7f9a24104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.142+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 shutdown_connections 2024-04-24T23:14:13.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.142+0000 7f9a28ba2640 1 -- 172.21.15.112:0/3300266904 wait complete. 2024-04-24T23:14:13.153 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:12 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3238587526' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:14:13.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:14:13.155 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:14:13.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:12 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3238587526' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:14:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.254+0000 7fc040bc1640 1 -- 172.21.15.112:0/1710443764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 msgr2=0x7fc03c106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.254+0000 7fc040bc1640 1 --2- 172.21.15.112:0/1710443764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 0x7fc03c106f30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc030009900 tx=0x7fc03002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 -- 172.21.15.112:0/1710443764 shutdown_connections 2024-04-24T23:14:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 --2- 172.21.15.112:0/1710443764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc03c107470 0x7fc03c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 --2- 172.21.15.112:0/1710443764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 0x7fc03c106f30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 --2- 172.21.15.112:0/1710443764 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c102210 0x7fc03c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 -- 172.21.15.112:0/1710443764 >> 172.21.15.112:0/1710443764 conn(0x7fc03c0fbc00 msgr2=0x7fc03c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 -- 172.21.15.112:0/1710443764 shutdown_connections 2024-04-24T23:14:13.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 -- 172.21.15.112:0/1710443764 wait complete. 2024-04-24T23:14:13.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 Processor -- start 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 -- start start 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc03c102210 0x7fc03c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 0x7fc03c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c107470 0x7fc03c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c10d540 con 0x7fc03c104b40 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c10d3c0 con 0x7fc03c107470 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.255+0000 7fc040bc1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c10d6c0 con 0x7fc03c102210 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 0x7fc03c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 0x7fc03c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56132/0 (socket says 172.21.15.112:56132) 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 -- 172.21.15.112:0/3517793084 learned_addr learned my addr 172.21.15.112:0/3517793084 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc03a59c640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc03c102210 0x7fc03c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc03ad9d640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c107470 0x7fc03c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 -- 172.21.15.112:0/3517793084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc03c102210 msgr2=0x7fc03c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc03c102210 0x7fc03c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 -- 172.21.15.112:0/3517793084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c107470 msgr2=0x7fc03c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c107470 0x7fc03c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 -- 172.21.15.112:0/3517793084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c19f7a0 con 0x7fc03c104b40 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc03a59c640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc03c102210 0x7fc03c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc03ad9d640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c107470 0x7fc03c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc039d9b640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 0x7fc03c1980c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc030006010 tx=0x7fc030004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc02b7fe640 1 -- 172.21.15.112:0/3517793084 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03003f030 con 0x7fc03c104b40 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c19fa30 con 0x7fc03c104b40 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc02b7fe640 1 -- 172.21.15.112:0/3517793084 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc030004490 con 0x7fc03c104b40 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c19ff70 con 0x7fc03c104b40 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.256+0000 7fc02b7fe640 1 -- 172.21.15.112:0/3517793084 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030049060 con 0x7fc03c104b40 2024-04-24T23:14:13.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.257+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c0fd6f0 con 0x7fc03c104b40 2024-04-24T23:14:13.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.258+0000 7fc02b7fe640 1 -- 172.21.15.112:0/3517793084 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc030042030 con 0x7fc03c104b40 2024-04-24T23:14:13.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.259+0000 7fc02b7fe640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc0240a2b90 0x7fc0240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.259+0000 7fc03a59c640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc0240a2b90 0x7fc0240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.259+0000 7fc02b7fe640 1 -- 172.21.15.112:0/3517793084 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc0300f0010 con 0x7fc03c104b40 2024-04-24T23:14:13.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.259+0000 7fc03a59c640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc0240a2b90 0x7fc0240a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00abe0 tx=0x7fc02c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.262+0000 7fc02b7fe640 1 -- 172.21.15.112:0/3517793084 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc0300bcc10 con 0x7fc03c104b40 2024-04-24T23:14:13.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:12 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3238587526' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:14:13.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.436+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc03c02d370 con 0x7fc03c104b40 2024-04-24T23:14:13.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.436+0000 7fc02b7fe640 1 -- 172.21.15.112:0/3517793084 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fc0300bc510 con 0x7fc03c104b40 2024-04-24T23:14:13.437 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:14:13.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc0240a2b90 msgr2=0x7fc0240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc0240a2b90 0x7fc0240a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc02c00abe0 tx=0x7fc02c00a5c0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 msgr2=0x7fc03c1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 0x7fc03c1980c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc030006010 tx=0x7fc030004270 comp rx=0 tx=0).stop 2024-04-24T23:14:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 shutdown_connections 2024-04-24T23:14:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc0240a2b90 0x7fc0240a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc03c107470 0x7fc03c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc03c104b40 0x7fc03c1980c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 --2- 172.21.15.112:0/3517793084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc03c102210 0x7fc03c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 >> 172.21.15.112:0/3517793084 conn(0x7fc03c0fbc00 msgr2=0x7fc03c102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 shutdown_connections 2024-04-24T23:14:13.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.438+0000 7fc040bc1640 1 -- 172.21.15.112:0/3517793084 wait complete. 2024-04-24T23:14:13.450 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:14:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/3901474505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc101880 msgr2=0x7ff0cc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/3901474505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc101880 0x7ff0cc101d00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009900 tx=0x7ff0b402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:13.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/3901474505 shutdown_connections 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/3901474505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc102240 0x7ff0cc108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/3901474505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc101880 0x7ff0cc101d00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/3901474505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc100690 0x7ff0cc100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/3901474505 >> 172.21.15.112:0/3901474505 conn(0x7ff0cc0fbc00 msgr2=0x7ff0cc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/3901474505 shutdown_connections 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.550+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/3901474505 wait complete. 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0d0fc8640 1 Processor -- start 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0d0fc8640 1 -- start start 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0d0fc8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc100690 0x7ff0cc06a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0d0fc8640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc101880 0x7ff0cc19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0d0fc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc102240 0x7ff0cc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0d0fc8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc10c440 con 0x7ff0cc102240 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0d0fc8640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc10c2c0 con 0x7ff0cc101880 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0d0fc8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc10c5c0 con 0x7ff0cc100690 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0cad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc102240 0x7ff0cc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0cad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc102240 0x7ff0cc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56150/0 (socket says 172.21.15.112:56150) 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0cad9d640 1 -- 172.21.15.112:0/2259481248 learned_addr learned my addr 172.21.15.112:0/2259481248 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0ca59c640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc100690 0x7ff0cc06a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0c3fff640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc101880 0x7ff0cc19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0cad9d640 1 -- 172.21.15.112:0/2259481248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc100690 msgr2=0x7ff0cc06a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.551+0000 7ff0cad9d640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc100690 0x7ff0cc06a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0cad9d640 1 -- 172.21.15.112:0/2259481248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc101880 msgr2=0x7ff0cc19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0cad9d640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc101880 0x7ff0cc19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0cad9d640 1 -- 172.21.15.112:0/2259481248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc19f840 con 0x7ff0cc102240 2024-04-24T23:14:13.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0ca59c640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc100690 0x7ff0cc06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:13.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0c3fff640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc101880 0x7ff0cc19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:13.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0cad9d640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc102240 0x7ff0cc19ef30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400cad0 tx=0x7ff0c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0c37fe640 1 -- 172.21.15.112:0/2259481248 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400ae20 con 0x7ff0cc102240 2024-04-24T23:14:13.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0c37fe640 1 -- 172.21.15.112:0/2259481248 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0c4004b20 con 0x7ff0cc102240 2024-04-24T23:14:13.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc19fad0 con 0x7ff0cc102240 2024-04-24T23:14:13.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0c37fe640 1 -- 172.21.15.112:0/2259481248 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400f780 con 0x7ff0cc102240 2024-04-24T23:14:13.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.552+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc1a0020 con 0x7ff0cc102240 2024-04-24T23:14:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.553+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0cc1079b0 con 0x7ff0cc102240 2024-04-24T23:14:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.554+0000 7ff0c37fe640 1 -- 172.21.15.112:0/2259481248 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff0c400f940 con 0x7ff0cc102240 2024-04-24T23:14:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.555+0000 7ff0c37fe640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0b80a2b90 0x7ff0b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.555+0000 7ff0ca59c640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0b80a2b90 0x7ff0b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.555+0000 7ff0c37fe640 1 -- 172.21.15.112:0/2259481248 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff0c40cf840 con 0x7ff0cc102240 2024-04-24T23:14:13.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.555+0000 7ff0ca59c640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0b80a2b90 0x7ff0b80a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc1016e0 tx=0x7ff0bc005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.558+0000 7ff0c37fe640 1 -- 172.21.15.112:0/2259481248 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff0c4014030 con 0x7ff0cc102240 2024-04-24T23:14:13.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.733+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff0cc02d3c0 con 0x7ff0cc102240 2024-04-24T23:14:13.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.734+0000 7ff0c37fe640 1 -- 172.21.15.112:0/2259481248 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7ff0c409c440 con 0x7ff0cc102240 2024-04-24T23:14:13.735 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0b80a2b90 msgr2=0x7ff0b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0b80a2b90 0x7ff0b80a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc1016e0 tx=0x7ff0bc005d30 comp rx=0 tx=0).stop 2024-04-24T23:14:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc102240 msgr2=0x7ff0cc19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc102240 0x7ff0cc19ef30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400cad0 tx=0x7ff0c400cf90 comp rx=0 tx=0).stop 2024-04-24T23:14:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 shutdown_connections 2024-04-24T23:14:13.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0b80a2b90 0x7ff0b80a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0cc102240 0x7ff0cc19ef30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0cc101880 0x7ff0cc19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 --2- 172.21.15.112:0/2259481248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0cc100690 0x7ff0cc06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 >> 172.21.15.112:0/2259481248 conn(0x7ff0cc0fbc00 msgr2=0x7ff0cc104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 shutdown_connections 2024-04-24T23:14:13.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.736+0000 7ff0d0fc8640 1 -- 172.21.15.112:0/2259481248 wait complete. 2024-04-24T23:14:13.748 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 -- 172.21.15.112:0/3307481434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414101890 msgr2=0x7f1414101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 --2- 172.21.15.112:0/3307481434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414101890 0x7f1414101d10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1408009ca0 tx=0x7f140802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 -- 172.21.15.112:0/3307481434 shutdown_connections 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 --2- 172.21.15.112:0/3307481434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1414102250 0x7f1414108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 --2- 172.21.15.112:0/3307481434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414101890 0x7f1414101d10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 --2- 172.21.15.112:0/3307481434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14141006a0 0x7f1414100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 -- 172.21.15.112:0/3307481434 >> 172.21.15.112:0/3307481434 conn(0x7f14140fbc10 msgr2=0x7f14140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 -- 172.21.15.112:0/3307481434 shutdown_connections 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.847+0000 7f14190e1640 1 -- 172.21.15.112:0/3307481434 wait complete. 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f14190e1640 1 Processor -- start 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f14190e1640 1 -- start start 2024-04-24T23:14:13.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f14190e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14141006a0 0x7f141406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f14190e1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1414101890 0x7f141419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f14190e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1414102250 0x7f141419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f14190e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f141410c450 con 0x7f14141006a0 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f14190e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f141410c2d0 con 0x7f1414102250 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f14190e1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f141410c5d0 con 0x7f1414101890 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.848+0000 7f1412d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14141006a0 0x7f141406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f1412d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14141006a0 0x7f141406a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56164/0 (socket says 172.21.15.112:56164) 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f1412d9d640 1 -- 172.21.15.112:0/2424361282 learned_addr learned my addr 172.21.15.112:0/2424361282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f141359e640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1414102250 0x7f141419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f141259c640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1414101890 0x7f141419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f1412d9d640 1 -- 172.21.15.112:0/2424361282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1414101890 msgr2=0x7f141419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f1412d9d640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1414101890 0x7f141419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f1412d9d640 1 -- 172.21.15.112:0/2424361282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1414102250 msgr2=0x7f141419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f1412d9d640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1414102250 0x7f141419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f1412d9d640 1 -- 172.21.15.112:0/2424361282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141419f870 con 0x7f14141006a0 2024-04-24T23:14:13.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f1412d9d640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14141006a0 0x7f141406a8b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c970 tx=0x7f140c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f13fbfff640 1 -- 172.21.15.112:0/2424361282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c004220 con 0x7f14141006a0 2024-04-24T23:14:13.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f13fbfff640 1 -- 172.21.15.112:0/2424361282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f140c0043c0 con 0x7f14141006a0 2024-04-24T23:14:13.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141419fb00 con 0x7f14141006a0 2024-04-24T23:14:13.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f13fbfff640 1 -- 172.21.15.112:0/2424361282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c004a00 con 0x7f14141006a0 2024-04-24T23:14:13.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.849+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14141a0000 con 0x7f14141006a0 2024-04-24T23:14:13.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.850+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1414107a40 con 0x7f14141006a0 2024-04-24T23:14:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.851+0000 7f13fbfff640 1 -- 172.21.15.112:0/2424361282 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f140c0079b0 con 0x7f14141006a0 2024-04-24T23:14:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.852+0000 7f13fbfff640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f13fc0a2b90 0x7f13fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.852+0000 7f141259c640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f13fc0a2b90 0x7f13fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.852+0000 7f13fbfff640 1 -- 172.21.15.112:0/2424361282 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f140c0cf1d0 con 0x7f14141006a0 2024-04-24T23:14:13.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.852+0000 7f141259c640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f13fc0a2b90 0x7f13fc0a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1408000f80 tx=0x7f1408005fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:13.856+0000 7f13fbfff640 1 -- 172.21.15.112:0/2424361282 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f140c014030 con 0x7f14141006a0 2024-04-24T23:14:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:13 smithi112 ceph-mon[29186]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:13 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3300266904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:13 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3517793084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:14.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:13 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2259481248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:14.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.034+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14140623e0 con 0x7f14141006a0 2024-04-24T23:14:14.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.034+0000 7f13fbfff640 1 -- 172.21.15.112:0/2424361282 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f140c09bdd0 con 0x7f14141006a0 2024-04-24T23:14:14.035 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:14:14.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f13fc0a2b90 msgr2=0x7f13fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f13fc0a2b90 0x7f13fc0a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1408000f80 tx=0x7f1408005fa0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14141006a0 msgr2=0x7f141406a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14141006a0 0x7f141406a8b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c970 tx=0x7f140c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 shutdown_connections 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f13fc0a2b90 0x7f13fc0a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1414102250 0x7f141419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1414101890 0x7f141419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 --2- 172.21.15.112:0/2424361282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14141006a0 0x7f141406a8b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.036+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 >> 172.21.15.112:0/2424361282 conn(0x7f14140fbc10 msgr2=0x7f1414104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.037+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 shutdown_connections 2024-04-24T23:14:14.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.037+0000 7f14190e1640 1 -- 172.21.15.112:0/2424361282 wait complete. 2024-04-24T23:14:14.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:14:14.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 -- 172.21.15.112:0/1974933911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c81006a0 msgr2=0x7ff8c8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 --2- 172.21.15.112:0/1974933911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c81006a0 0x7ff8c8100ac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009900 tx=0x7ff8bc02a410 comp rx=0 tx=0).stop 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 -- 172.21.15.112:0/1974933911 shutdown_connections 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 --2- 172.21.15.112:0/1974933911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c8102250 0x7ff8c8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 --2- 172.21.15.112:0/1974933911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8c8101890 0x7ff8c8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 --2- 172.21.15.112:0/1974933911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c81006a0 0x7ff8c8100ac0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 -- 172.21.15.112:0/1974933911 >> 172.21.15.112:0/1974933911 conn(0x7ff8c80fbc30 msgr2=0x7ff8c80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 -- 172.21.15.112:0/1974933911 shutdown_connections 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.148+0000 7ff8ceded640 1 -- 172.21.15.112:0/1974933911 wait complete. 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ceded640 1 Processor -- start 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ceded640 1 -- start start 2024-04-24T23:14:14.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ceded640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81006a0 0x7ff8c8197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ceded640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8c8101890 0x7ff8c8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ceded640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c8102250 0x7ff8c819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ceded640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810c4b0 con 0x7ff8c8102250 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ceded640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810c330 con 0x7ff8c8101890 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ceded640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c810c630 con 0x7ff8c81006a0 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8cd38a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c8102250 0x7ff8c819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8cd38a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c8102250 0x7ff8c819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56188/0 (socket says 172.21.15.112:56188) 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8cd38a640 1 -- 172.21.15.112:0/3867558445 learned_addr learned my addr 172.21.15.112:0/3867558445 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8ccb89640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81006a0 0x7ff8c8197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8c7fff640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8c8101890 0x7ff8c8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.149+0000 7ff8cd38a640 1 -- 172.21.15.112:0/3867558445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81006a0 msgr2=0x7ff8c8197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8cd38a640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81006a0 0x7ff8c8197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8cd38a640 1 -- 172.21.15.112:0/3867558445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8c8101890 msgr2=0x7ff8c8198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8cd38a640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8c8101890 0x7ff8c8198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8cd38a640 1 -- 172.21.15.112:0/3867558445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c819f860 con 0x7ff8c8102250 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8c7fff640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8c8101890 0x7ff8c8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8cd38a640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c8102250 0x7ff8c819f1c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b970 tx=0x7ff8c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8c5ffb640 1 -- 172.21.15.112:0/3867558445 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0019070 con 0x7ff8c8102250 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c819fb50 con 0x7ff8c8102250 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8c5ffb640 1 -- 172.21.15.112:0/3867558445 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8c000cb40 con 0x7ff8c8102250 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8c5ffb640 1 -- 172.21.15.112:0/3867558445 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c0007700 con 0x7ff8c8102250 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.150+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c81a0050 con 0x7ff8c8102250 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.151+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c81a0280 con 0x7ff8c8102250 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.152+0000 7ff8c5ffb640 1 -- 172.21.15.112:0/3867558445 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff8c001c070 con 0x7ff8c8102250 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.152+0000 7ff8c5ffb640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8b80a2b90 0x7ff8b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.153+0000 7ff8ccb89640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8b80a2b90 0x7ff8b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.153+0000 7ff8c5ffb640 1 -- 172.21.15.112:0/3867558445 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff8c00cea80 con 0x7ff8c8102250 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.153+0000 7ff8ccb89640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8b80a2b90 0x7ff8b80a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc031570 tx=0x7ff8bc005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.157+0000 7ff8c5ffb640 1 -- 172.21.15.112:0/3867558445 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8c009b680 con 0x7ff8c8102250 2024-04-24T23:14:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:13 smithi187 ceph-mon[33292]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:13 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3300266904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:13 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3517793084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:13 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2259481248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:13 smithi150 ceph-mon[32247]: pgmap v345: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:13 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3300266904' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:13 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3517793084' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:13 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2259481248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:14.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.334+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff8c81a0450 con 0x7ff8c8102250 2024-04-24T23:14:14.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.335+0000 7ff8c5ffb640 1 -- 172.21.15.112:0/3867558445 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7ff8c009af80 con 0x7ff8c8102250 2024-04-24T23:14:14.336 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8b80a2b90 msgr2=0x7ff8b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8b80a2b90 0x7ff8b80a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc031570 tx=0x7ff8bc005be0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c8102250 msgr2=0x7ff8c819f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c8102250 0x7ff8c819f1c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b970 tx=0x7ff8c000be30 comp rx=0 tx=0).stop 2024-04-24T23:14:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 shutdown_connections 2024-04-24T23:14:14.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8b80a2b90 0x7ff8b80a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8c8102250 0x7ff8c819f1c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8c8101890 0x7ff8c8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 --2- 172.21.15.112:0/3867558445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8c81006a0 0x7ff8c8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 >> 172.21.15.112:0/3867558445 conn(0x7ff8c80fbc30 msgr2=0x7ff8c8069d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 shutdown_connections 2024-04-24T23:14:14.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.337+0000 7ff8ceded640 1 -- 172.21.15.112:0/3867558445 wait complete. 2024-04-24T23:14:14.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.446+0000 7fef8b39e640 1 -- 172.21.15.112:0/4068205486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 msgr2=0x7fef84101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.446+0000 7fef8b39e640 1 --2- 172.21.15.112:0/4068205486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 0x7fef84101b30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fef80009900 tx=0x7fef8002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.446+0000 7fef8b39e640 1 -- 172.21.15.112:0/4068205486 shutdown_connections 2024-04-24T23:14:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.446+0000 7fef8b39e640 1 --2- 172.21.15.112:0/4068205486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef84102070 0x7fef84108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 --2- 172.21.15.112:0/4068205486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 0x7fef84101b30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 --2- 172.21.15.112:0/4068205486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef841004c0 0x7fef841008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 -- 172.21.15.112:0/4068205486 >> 172.21.15.112:0/4068205486 conn(0x7fef840fba50 msgr2=0x7fef840fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 -- 172.21.15.112:0/4068205486 shutdown_connections 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 -- 172.21.15.112:0/4068205486 wait complete. 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 Processor -- start 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 -- start start 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef841004c0 0x7fef841979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 0x7fef84197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef84102070 0x7fef8419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410d3e0 con 0x7fef841016b0 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410d260 con 0x7fef841004c0 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.447+0000 7fef8b39e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fef8410d560 con 0x7fef84102070 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 0x7fef84197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 0x7fef84197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56196/0 (socket says 172.21.15.112:56196) 2024-04-24T23:14:14.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 -- 172.21.15.112:0/2061131539 learned_addr learned my addr 172.21.15.112:0/2061131539 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef8a39c640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef841004c0 0x7fef841979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 -- 172.21.15.112:0/2061131539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef84102070 msgr2=0x7fef8419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef84102070 0x7fef8419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 -- 172.21.15.112:0/2061131539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef841004c0 msgr2=0x7fef841979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef841004c0 0x7fef841979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 -- 172.21.15.112:0/2061131539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8419f610 con 0x7fef841016b0 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef89b9b640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 0x7fef84197f30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef80009a20 tx=0x7fef80004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef777fe640 1 -- 172.21.15.112:0/2061131539 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8003f030 con 0x7fef841016b0 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef777fe640 1 -- 172.21.15.112:0/2061131539 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fef80003d00 con 0x7fef841016b0 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef777fe640 1 -- 172.21.15.112:0/2061131539 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80035070 con 0x7fef841016b0 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8419f8a0 con 0x7fef841016b0 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.448+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef8419fe60 con 0x7fef841016b0 2024-04-24T23:14:14.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.449+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef84107780 con 0x7fef841016b0 2024-04-24T23:14:14.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.450+0000 7fef777fe640 1 -- 172.21.15.112:0/2061131539 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fef80042030 con 0x7fef841016b0 2024-04-24T23:14:14.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.451+0000 7fef777fe640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fef700a2c10 0x7fef700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.451+0000 7fef8a39c640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fef700a2c10 0x7fef700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.451+0000 7fef777fe640 1 -- 172.21.15.112:0/2061131539 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fef800ef9c0 con 0x7fef841016b0 2024-04-24T23:14:14.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.451+0000 7fef8a39c640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fef700a2c10 0x7fef700a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef84101510 tx=0x7fef7800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.455+0000 7fef777fe640 1 -- 172.21.15.112:0/2061131539 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fef800f1e20 con 0x7fef841016b0 2024-04-24T23:14:14.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.669+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef84062340 con 0x7fef841016b0 2024-04-24T23:14:14.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.670+0000 7fef777fe640 1 -- 172.21.15.112:0/2061131539 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11947 (secure 0 0 0) 0x7fef800bc5c0 con 0x7fef841016b0 2024-04-24T23:14:14.671 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:14.671 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:12:55.876908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24482,24490,24425,24424]},"id":1}]} 2024-04-24T23:14:14.672 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-04-24T23:14:14.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fef700a2c10 msgr2=0x7fef700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fef700a2c10 0x7fef700a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef84101510 tx=0x7fef7800b390 comp rx=0 tx=0).stop 2024-04-24T23:14:14.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 msgr2=0x7fef84197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 0x7fef84197f30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef80009a20 tx=0x7fef80004430 comp rx=0 tx=0).stop 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 shutdown_connections 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fef700a2c10 0x7fef700a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef84102070 0x7fef8419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef841016b0 0x7fef84197f30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 --2- 172.21.15.112:0/2061131539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef841004c0 0x7fef841979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 >> 172.21.15.112:0/2061131539 conn(0x7fef840fba50 msgr2=0x7fef84104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 shutdown_connections 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.672+0000 7fef8b39e640 1 -- 172.21.15.112:0/2061131539 wait complete. 2024-04-24T23:14:14.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 -- 172.21.15.112:0/1205462827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 msgr2=0x7f69c0101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 --2- 172.21.15.112:0/1205462827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 0x7f69c0101bd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009900 tx=0x7f69a802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 -- 172.21.15.112:0/1205462827 shutdown_connections 2024-04-24T23:14:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 --2- 172.21.15.112:0/1205462827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c0103360 0x7f69c0109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 --2- 172.21.15.112:0/1205462827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c01029a0 0x7f69c0102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 --2- 172.21.15.112:0/1205462827 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 0x7f69c0101bd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 -- 172.21.15.112:0/1205462827 >> 172.21.15.112:0/1205462827 conn(0x7f69c00fcd40 msgr2=0x7f69c00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 -- 172.21.15.112:0/1205462827 shutdown_connections 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.781+0000 7f69c6876640 1 -- 172.21.15.112:0/1205462827 wait complete. 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69c6876640 1 Processor -- start 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69c6876640 1 -- start start 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69c6876640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 0x7f69c00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69c6876640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c01029a0 0x7f69c0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69c6876640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c0103360 0x7f69c0074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69c6876640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69c010d560 con 0x7f69c01017b0 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69c6876640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f69c010d3e0 con 0x7f69c0103360 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69c6876640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69c010d6e0 con 0x7f69c01029a0 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 0x7f69c00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69bffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 0x7f69c00754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56222/0 (socket says 172.21.15.112:56222) 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.782+0000 7f69bffff640 1 -- 172.21.15.112:0/501260011 learned_addr learned my addr 172.21.15.112:0/501260011 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69c4e13640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c0103360 0x7f69c0074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bffff640 1 -- 172.21.15.112:0/501260011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c01029a0 msgr2=0x7f69c0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bf7fe640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c01029a0 0x7f69c0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bffff640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c01029a0 0x7f69c0073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bffff640 1 -- 172.21.15.112:0/501260011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c0103360 msgr2=0x7f69c0074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bffff640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c0103360 0x7f69c0074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bffff640 1 -- 172.21.15.112:0/501260011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c0074900 con 0x7f69c01017b0 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69c4e13640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c0103360 0x7f69c0074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bffff640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 0x7f69c00754f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009230 tx=0x7f69a8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bd7fa640 1 -- 172.21.15.112:0/501260011 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a803f030 con 0x7f69c01017b0 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c01a1c70 con 0x7f69c01017b0 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bd7fa640 1 -- 172.21.15.112:0/501260011 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f69a8003d00 con 0x7f69c01017b0 2024-04-24T23:14:14.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69bd7fa640 1 -- 172.21.15.112:0/501260011 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8035070 con 0x7f69c01017b0 2024-04-24T23:14:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.783+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c01a21c0 con 0x7f69c01017b0 2024-04-24T23:14:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.784+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c0108ad0 con 0x7f69c01017b0 2024-04-24T23:14:14.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.785+0000 7f69bd7fa640 1 -- 172.21.15.112:0/501260011 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f69a8042030 con 0x7f69c01017b0 2024-04-24T23:14:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.785+0000 7f69bd7fa640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f69ac0a2b90 0x7f69ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.786+0000 7f69bf7fe640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f69ac0a2b90 0x7f69ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.786+0000 7f69bd7fa640 1 -- 172.21.15.112:0/501260011 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f69a80efb90 con 0x7f69c01017b0 2024-04-24T23:14:14.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.786+0000 7f69bf7fe640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f69ac0a2b90 0x7f69ac0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f69c0075260 tx=0x7f69b0005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.790+0000 7f69bd7fa640 1 -- 172.21.15.112:0/501260011 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f69a80bc790 con 0x7f69c01017b0 2024-04-24T23:14:14.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.970+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69c0062370 con 0x7f69c01017b0 2024-04-24T23:14:14.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.972+0000 7f69bd7fa640 1 -- 172.21.15.112:0/501260011 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24714 (secure 0 0 0) 0x7f69a80bc090 con 0x7f69c01017b0 2024-04-24T23:14:14.973 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:14.973 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":92,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:13:27.938988+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f69ac0a2b90 msgr2=0x7f69ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f69ac0a2b90 0x7f69ac0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f69c0075260 tx=0x7f69b0005c60 comp rx=0 tx=0).stop 2024-04-24T23:14:14.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 msgr2=0x7f69c00754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 0x7f69c00754f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009230 tx=0x7f69a8004430 comp rx=0 tx=0).stop 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 shutdown_connections 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f69ac0a2b90 0x7f69ac0a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f69c0103360 0x7f69c0074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69c01029a0 0x7f69c0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 --2- 172.21.15.112:0/501260011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69c01017b0 0x7f69c00754f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 >> 172.21.15.112:0/501260011 conn(0x7f69c00fcd40 msgr2=0x7f69c0105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 shutdown_connections 2024-04-24T23:14:14.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:14.974+0000 7f69c6876640 1 -- 172.21.15.112:0/501260011 wait complete. 2024-04-24T23:14:14.985 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:14 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2424361282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:14.985 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:14 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3867558445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:14.985 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:14 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2061131539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:14.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:14:14.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:14:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 -- 172.21.15.112:0/1837004368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc0ffa50 msgr2=0x7f08fc0ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 --2- 172.21.15.112:0/1837004368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc0ffa50 0x7f08fc0ffe70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009930 tx=0x7f08ec02a420 comp rx=0 tx=0).stop 2024-04-24T23:14:15.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 -- 172.21.15.112:0/1837004368 shutdown_connections 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 --2- 172.21.15.112:0/1837004368 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08fc1033e0 0x7f08fc109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 --2- 172.21.15.112:0/1837004368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08fc102a20 0x7f08fc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 --2- 172.21.15.112:0/1837004368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc0ffa50 0x7f08fc0ffe70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 -- 172.21.15.112:0/1837004368 >> 172.21.15.112:0/1837004368 conn(0x7f08fc0fb380 msgr2=0x7f08fc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 -- 172.21.15.112:0/1837004368 shutdown_connections 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.088+0000 7f0902970640 1 -- 172.21.15.112:0/1837004368 wait complete. 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 Processor -- start 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 -- start start 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08fc0ffa50 0x7f08fc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc102a20 0x7f08fc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08fc1033e0 0x7f08fc19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc10d5c0 con 0x7f08fc102a20 2024-04-24T23:14:15.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc10d440 con 0x7f08fc0ffa50 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08fc10d740 con 0x7f08fc1033e0 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc102a20 0x7f08fc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc102a20 0x7f08fc1980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56234/0 (socket says 172.21.15.112:56234) 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 -- 172.21.15.112:0/2195251325 learned_addr learned my addr 172.21.15.112:0/2195251325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08fbfff640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08fc0ffa50 0x7f08fc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0900f0d640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08fc1033e0 0x7f08fc19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 -- 172.21.15.112:0/2195251325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08fc1033e0 msgr2=0x7f08fc19f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08fc1033e0 0x7f08fc19f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 -- 172.21.15.112:0/2195251325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08fc0ffa50 msgr2=0x7f08fc197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08fc0ffa50 0x7f08fc197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 -- 172.21.15.112:0/2195251325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08fc19f790 con 0x7f08fc102a20 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08fbfff640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08fc0ffa50 0x7f08fc197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f3fff640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc102a20 0x7f08fc1980b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f08e400c920 tx=0x7f08e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:15.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f9ffb640 1 -- 172.21.15.112:0/2195251325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4007c70 con 0x7f08fc102a20 2024-04-24T23:14:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f9ffb640 1 -- 172.21.15.112:0/2195251325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f08e4007e10 con 0x7f08fc102a20 2024-04-24T23:14:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08fc19fa80 con 0x7f08fc102a20 2024-04-24T23:14:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f08f9ffb640 1 -- 172.21.15.112:0/2195251325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4018940 con 0x7f08fc102a20 2024-04-24T23:14:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.089+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08fc19ffd0 con 0x7f08fc102a20 2024-04-24T23:14:15.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.090+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08fc108af0 con 0x7f08fc102a20 2024-04-24T23:14:15.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.091+0000 7f08f9ffb640 1 -- 172.21.15.112:0/2195251325 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f08e4018b20 con 0x7f08fc102a20 2024-04-24T23:14:15.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.092+0000 7f08f9ffb640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e80a2b90 0x7f08e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.092+0000 7f08fbfff640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e80a2b90 0x7f08e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.092+0000 7f08f9ffb640 1 -- 172.21.15.112:0/2195251325 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f08e40cf5f0 con 0x7f08fc102a20 2024-04-24T23:14:15.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.092+0000 7f08fbfff640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e80a2b90 0x7f08e80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009900 tx=0x7f08ec005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:15.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.097+0000 7f08f9ffb640 1 -- 172.21.15.112:0/2195251325 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08e4014030 con 0x7f08fc102a20 2024-04-24T23:14:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:14 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2424361282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:14 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3867558445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:15.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:14 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2061131539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:15.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.271+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08fc062340 con 0x7f08fc102a20 2024-04-24T23:14:15.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.272+0000 7f08f9ffb640 1 -- 172.21.15.112:0/2195251325 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f08e400de20 con 0x7f08fc102a20 2024-04-24T23:14:15.273 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:14:15.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e80a2b90 msgr2=0x7f08e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e80a2b90 0x7f08e80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009900 tx=0x7f08ec005be0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc102a20 msgr2=0x7f08fc1980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc102a20 0x7f08fc1980b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f08e400c920 tx=0x7f08e400cde0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 shutdown_connections 2024-04-24T23:14:15.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f08e80a2b90 0x7f08e80a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08fc1033e0 0x7f08fc19f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08fc102a20 0x7f08fc1980b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 --2- 172.21.15.112:0/2195251325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08fc0ffa50 0x7f08fc197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 >> 172.21.15.112:0/2195251325 conn(0x7f08fc0fb380 msgr2=0x7f08fc105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:15.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 shutdown_connections 2024-04-24T23:14:15.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.274+0000 7f0902970640 1 -- 172.21.15.112:0/2195251325 wait complete. 2024-04-24T23:14:15.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:14:15.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:14 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2424361282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:15.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:14 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3867558445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:15.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:14 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2061131539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:15.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 -- 172.21.15.112:0/3515333511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 msgr2=0x7f4754108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 --2- 172.21.15.112:0/3515333511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 0x7f4754108240 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f474c009ca0 tx=0x7f474c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 -- 172.21.15.112:0/3515333511 shutdown_connections 2024-04-24T23:14:15.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 --2- 172.21.15.112:0/3515333511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 0x7f4754108240 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 --2- 172.21.15.112:0/3515333511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4754100f60 0x7f47541013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 --2- 172.21.15.112:0/3515333511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47540ffe10 0x7f4754100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 -- 172.21.15.112:0/3515333511 >> 172.21.15.112:0/3515333511 conn(0x7f47540fb380 msgr2=0x7f47540fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:15.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 -- 172.21.15.112:0/3515333511 shutdown_connections 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 -- 172.21.15.112:0/3515333511 wait complete. 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 Processor -- start 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 -- start start 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47540ffe10 0x7f4754197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.386+0000 7f475b15c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4754100f60 0x7f4754198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f475b15c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 0x7f475419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f475b15c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f475410b3e0 con 0x7f4754101920 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f475b15c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f475410b260 con 0x7f47540ffe10 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f475b15c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f475410b560 con 0x7f4754100f60 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 0x7f475419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 0x7f475419f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56256/0 (socket says 172.21.15.112:56256) 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 -- 172.21.15.112:0/289670889 learned_addr learned my addr 172.21.15.112:0/289670889 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:15.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f4758ef8640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47540ffe10 0x7f4754197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f4753fff640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4754100f60 0x7f4754198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 -- 172.21.15.112:0/289670889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4754100f60 msgr2=0x7f4754198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4754100f60 0x7f4754198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 -- 172.21.15.112:0/289670889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47540ffe10 msgr2=0x7f4754197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47540ffe10 0x7f4754197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 -- 172.21.15.112:0/289670889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475419f790 con 0x7f4754101920 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f47596f9640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 0x7f475419f120 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f474c02f540 tx=0x7f474c031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f4751ffb640 1 -- 172.21.15.112:0/289670889 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c003b70 con 0x7f4754101920 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f4751ffb640 1 -- 172.21.15.112:0/289670889 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f474c003d10 con 0x7f4754101920 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475419fa20 con 0x7f4754101920 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.387+0000 7f4751ffb640 1 -- 172.21.15.112:0/289670889 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c02fc00 con 0x7f4754101920 2024-04-24T23:14:15.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.388+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475419ffc0 con 0x7f4754101920 2024-04-24T23:14:15.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.388+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47541a01f0 con 0x7f4754101920 2024-04-24T23:14:15.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.390+0000 7f4751ffb640 1 -- 172.21.15.112:0/289670889 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f474c0046b0 con 0x7f4754101920 2024-04-24T23:14:15.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.390+0000 7f4751ffb640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47440a2c10 0x7f47440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.390+0000 7f4758ef8640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47440a2c10 0x7f47440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.390+0000 7f4751ffb640 1 -- 172.21.15.112:0/289670889 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f474c0f0900 con 0x7f4754101920 2024-04-24T23:14:15.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.391+0000 7f4758ef8640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47440a2c10 0x7f47440a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f474800ac40 tx=0x7f4748010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:15.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.394+0000 7f4751ffb640 1 -- 172.21.15.112:0/289670889 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f474c0bd500 con 0x7f4754101920 2024-04-24T23:14:15.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.572+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47541a03c0 con 0x7f4754101920 2024-04-24T23:14:15.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.572+0000 7f4751ffb640 1 -- 172.21.15.112:0/289670889 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f474c0bce00 con 0x7f4754101920 2024-04-24T23:14:15.573 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47440a2c10 msgr2=0x7f47440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47440a2c10 0x7f47440a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f474800ac40 tx=0x7f4748010040 comp rx=0 tx=0).stop 2024-04-24T23:14:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 msgr2=0x7f475419f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 0x7f475419f120 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f474c02f540 tx=0x7f474c031750 comp rx=0 tx=0).stop 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 shutdown_connections 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f47440a2c10 0x7f47440a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4754101920 0x7f475419f120 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4754100f60 0x7f4754198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 --2- 172.21.15.112:0/289670889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47540ffe10 0x7f4754197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 >> 172.21.15.112:0/289670889 conn(0x7f47540fb380 msgr2=0x7f4754106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 shutdown_connections 2024-04-24T23:14:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.574+0000 7f475b15c640 1 -- 172.21.15.112:0/289670889 wait complete. 2024-04-24T23:14:15.587 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:14:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 -- 172.21.15.112:0/3172186973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c8102250 msgr2=0x7f40c8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 --2- 172.21.15.112:0/3172186973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c8102250 0x7f40c8108ae0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009ca0 tx=0x7f40c002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 -- 172.21.15.112:0/3172186973 shutdown_connections 2024-04-24T23:14:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 --2- 172.21.15.112:0/3172186973 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c8102250 0x7f40c8108ae0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 --2- 172.21.15.112:0/3172186973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c8101890 0x7f40c8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 --2- 172.21.15.112:0/3172186973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40c81006a0 0x7f40c8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 -- 172.21.15.112:0/3172186973 >> 172.21.15.112:0/3172186973 conn(0x7f40c80fbc30 msgr2=0x7f40c80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 -- 172.21.15.112:0/3172186973 shutdown_connections 2024-04-24T23:14:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 -- 172.21.15.112:0/3172186973 wait complete. 2024-04-24T23:14:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 Processor -- start 2024-04-24T23:14:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.687+0000 7f40ceb09640 1 -- start start 2024-04-24T23:14:15.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40ceb09640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c81006a0 0x7f40c8104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40ceb09640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40c8101890 0x7f40c8102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40ceb09640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c8102250 0x7f40c81031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40ceb09640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40c810c450 con 0x7f40c81006a0 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40ceb09640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f40c810c2d0 con 0x7f40c8101890 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40ceb09640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40c810c5d0 con 0x7f40c8102250 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c81006a0 0x7f40c8104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c81006a0 0x7f40c8104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56274/0 (socket says 172.21.15.112:56274) 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 -- 172.21.15.112:0/1614284056 learned_addr learned my addr 172.21.15.112:0/1614284056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cd0a6640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c8102250 0x7f40c81031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40c7fff640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40c8101890 0x7f40c8102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 -- 172.21.15.112:0/1614284056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c8102250 msgr2=0x7f40c81031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c8102250 0x7f40c81031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 -- 172.21.15.112:0/1614284056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40c8101890 msgr2=0x7f40c8102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40c8101890 0x7f40c8102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 -- 172.21.15.112:0/1614284056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c8103a40 con 0x7f40c81006a0 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40cc8a5640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c81006a0 0x7f40c8104630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f40b800c970 tx=0x7f40b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40c5ffb640 1 -- 172.21.15.112:0/1614284056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8004220 con 0x7f40c81006a0 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40c5ffb640 1 -- 172.21.15.112:0/1614284056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f40b80043c0 con 0x7f40c81006a0 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c81a1bf0 con 0x7f40c81006a0 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.688+0000 7f40c5ffb640 1 -- 172.21.15.112:0/1614284056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8004a00 con 0x7f40c81006a0 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.689+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c81a20f0 con 0x7f40c81006a0 2024-04-24T23:14:15.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.690+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c81a2320 con 0x7f40c81006a0 2024-04-24T23:14:15.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.690+0000 7f40c5ffb640 1 -- 172.21.15.112:0/1614284056 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f40b80079b0 con 0x7f40c81006a0 2024-04-24T23:14:15.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.691+0000 7f40c5ffb640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40b40a2b90 0x7f40b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.691+0000 7f40c7fff640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40b40a2b90 0x7f40b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.691+0000 7f40c5ffb640 1 -- 172.21.15.112:0/1614284056 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f40b80cf180 con 0x7f40c81006a0 2024-04-24T23:14:15.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.691+0000 7f40c7fff640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40b40a2b90 0x7f40b40a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f40c81043a0 tx=0x7f40b0005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:15.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.695+0000 7f40c5ffb640 1 -- 172.21.15.112:0/1614284056 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f40b8014030 con 0x7f40c81006a0 2024-04-24T23:14:15.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.869+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40c81a24f0 con 0x7f40c81006a0 2024-04-24T23:14:15.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.869+0000 7f40c5ffb640 1 -- 172.21.15.112:0/1614284056 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f40b809bd80 con 0x7f40c81006a0 2024-04-24T23:14:15.870 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:14:15.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40b40a2b90 msgr2=0x7f40b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40b40a2b90 0x7f40b40a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f40c81043a0 tx=0x7f40b0005cc0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c81006a0 msgr2=0x7f40c8104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c81006a0 0x7f40c8104630 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f40b800c970 tx=0x7f40b800ce30 comp rx=0 tx=0).stop 2024-04-24T23:14:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 shutdown_connections 2024-04-24T23:14:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40b40a2b90 0x7f40b40a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40c8102250 0x7f40c81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40c8101890 0x7f40c8102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 --2- 172.21.15.112:0/1614284056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40c81006a0 0x7f40c8104630 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 >> 172.21.15.112:0/1614284056 conn(0x7f40c80fbc30 msgr2=0x7f40c8069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 shutdown_connections 2024-04-24T23:14:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.871+0000 7f40ceb09640 1 -- 172.21.15.112:0/1614284056 wait complete. 2024-04-24T23:14:15.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:14:15.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 -- 172.21.15.112:0/3064843015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 msgr2=0x7f2250108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 --2- 172.21.15.112:0/3064843015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 0x7f2250108ac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2248009ca0 tx=0x7f224802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 -- 172.21.15.112:0/3064843015 shutdown_connections 2024-04-24T23:14:15.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 --2- 172.21.15.112:0/3064843015 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 0x7f2250108ac0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 --2- 172.21.15.112:0/3064843015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2250101870 0x7f2250101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 --2- 172.21.15.112:0/3064843015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2250100680 0x7f2250100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 -- 172.21.15.112:0/3064843015 >> 172.21.15.112:0/3064843015 conn(0x7f22500fbbf0 msgr2=0x7f22500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:15.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 -- 172.21.15.112:0/3064843015 shutdown_connections 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.983+0000 7f22550bd640 1 -- 172.21.15.112:0/3064843015 wait complete. 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f22550bd640 1 Processor -- start 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f22550bd640 1 -- start start 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f22550bd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2250100680 0x7f22500694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f22550bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2250101870 0x7f225019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f22550bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 0x7f225019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f22550bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f225010c490 con 0x7f2250102230 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f22550bd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f225010c310 con 0x7f2250100680 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f22550bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f225010c610 con 0x7f2250101870 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f224f59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 0x7f225019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f224f59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 0x7f225019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56296/0 (socket says 172.21.15.112:56296) 2024-04-24T23:14:15.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f224f59e640 1 -- 172.21.15.112:0/3348906453 learned_addr learned my addr 172.21.15.112:0/3348906453 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.984+0000 7f224ed9d640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2250100680 0x7f22500694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f224e59c640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2250101870 0x7f225019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f224f59e640 1 -- 172.21.15.112:0/3348906453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2250101870 msgr2=0x7f225019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f224f59e640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2250101870 0x7f225019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f224f59e640 1 -- 172.21.15.112:0/3348906453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2250100680 msgr2=0x7f22500694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f224f59e640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2250100680 0x7f22500694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f224f59e640 1 -- 172.21.15.112:0/3348906453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225019f730 con 0x7f2250102230 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f224f59e640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 0x7f225019ee70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2248009dc0 tx=0x7f224802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f2237fff640 1 -- 172.21.15.112:0/3348906453 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224802fe40 con 0x7f2250102230 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225019f9c0 con 0x7f2250102230 2024-04-24T23:14:15.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f2237fff640 1 -- 172.21.15.112:0/3348906453 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2248030ac0 con 0x7f2250102230 2024-04-24T23:14:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f2237fff640 1 -- 172.21.15.112:0/3348906453 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248046bb0 con 0x7f2250102230 2024-04-24T23:14:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.985+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225019ffe0 con 0x7f2250102230 2024-04-24T23:14:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.986+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2250107940 con 0x7f2250102230 2024-04-24T23:14:15.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.987+0000 7f2237fff640 1 -- 172.21.15.112:0/3348906453 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2248030c70 con 0x7f2250102230 2024-04-24T23:14:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.987+0000 7f2237fff640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f22380a2b90 0x7f22380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.988+0000 7f224ed9d640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f22380a2b90 0x7f22380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.988+0000 7f2237fff640 1 -- 172.21.15.112:0/3348906453 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f22480f0ab0 con 0x7f2250102230 2024-04-24T23:14:15.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.988+0000 7f224ed9d640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f22380a2b90 0x7f22380a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f22501016d0 tx=0x7f2240005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:15.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:15.992+0000 7f2237fff640 1 -- 172.21.15.112:0/3348906453 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f22480bd6b0 con 0x7f2250102230 2024-04-24T23:14:16.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.173+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2250062340 con 0x7f2250102230 2024-04-24T23:14:16.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.174+0000 7f2237fff640 1 -- 172.21.15.112:0/3348906453 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f22480bcfb0 con 0x7f2250102230 2024-04-24T23:14:16.175 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f22380a2b90 msgr2=0x7f22380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f22380a2b90 0x7f22380a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f22501016d0 tx=0x7f2240005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 msgr2=0x7f225019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 0x7f225019ee70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2248009dc0 tx=0x7f224802f840 comp rx=0 tx=0).stop 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 shutdown_connections 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f22380a2b90 0x7f22380a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2250102230 0x7f225019ee70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2250101870 0x7f225019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 --2- 172.21.15.112:0/3348906453 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2250100680 0x7f22500694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 >> 172.21.15.112:0/3348906453 conn(0x7f22500fbbf0 msgr2=0x7f2250104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:16.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 shutdown_connections 2024-04-24T23:14:16.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.176+0000 7f22550bd640 1 -- 172.21.15.112:0/3348906453 wait complete. 2024-04-24T23:14:16.187 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:15 smithi112 ceph-mon[29186]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:16.187 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:15 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/501260011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:16.187 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:15 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2195251325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.187 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:15 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/289670889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.187 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:15 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1614284056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:15 smithi187 ceph-mon[33292]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:15 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/501260011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:15 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2195251325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:15 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/289670889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:15 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1614284056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 -- 172.21.15.112:0/3654234360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 msgr2=0x7f85d8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 --2- 172.21.15.112:0/3654234360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 0x7f85d8109c70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009cd0 tx=0x7f85d002a8e0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 -- 172.21.15.112:0/3654234360 shutdown_connections 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 --2- 172.21.15.112:0/3654234360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 0x7f85d8109c70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 --2- 172.21.15.112:0/3654234360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85d8102a20 0x7f85d8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 --2- 172.21.15.112:0/3654234360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d80ffa50 0x7f85d80ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 -- 172.21.15.112:0/3654234360 >> 172.21.15.112:0/3654234360 conn(0x7f85d80fb380 msgr2=0x7f85d80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 -- 172.21.15.112:0/3654234360 shutdown_connections 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.285+0000 7f85dd929640 1 -- 172.21.15.112:0/3654234360 wait complete. 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85dd929640 1 Processor -- start 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85dd929640 1 -- start start 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85dd929640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85d80ffa50 0x7f85d8197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85dd929640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8102a20 0x7f85d8198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85dd929640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 0x7f85d819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85dd929640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85d810d5c0 con 0x7f85d81033e0 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85dd929640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f85d810d440 con 0x7f85d8102a20 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85dd929640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85d810d740 con 0x7f85d80ffa50 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 0x7f85d819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 0x7f85d819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56318/0 (socket says 172.21.15.112:56318) 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d77fe640 1 -- 172.21.15.112:0/2021941230 learned_addr learned my addr 172.21.15.112:0/2021941230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d77fe640 1 -- 172.21.15.112:0/2021941230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85d80ffa50 msgr2=0x7f85d8197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d77fe640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85d80ffa50 0x7f85d8197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d77fe640 1 -- 172.21.15.112:0/2021941230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8102a20 msgr2=0x7f85d8198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d67fc640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8102a20 0x7f85d8198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d77fe640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8102a20 0x7f85d8198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.286+0000 7f85d77fe640 1 -- 172.21.15.112:0/2021941230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d819f7f0 con 0x7f85d81033e0 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.287+0000 7f85d77fe640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 0x7f85d819f180 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009df0 tx=0x7f85d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.287+0000 7f85dc927640 1 -- 172.21.15.112:0/2021941230 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0004450 con 0x7f85d81033e0 2024-04-24T23:14:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.287+0000 7f85dc927640 1 -- 172.21.15.112:0/2021941230 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f85d00045f0 con 0x7f85d81033e0 2024-04-24T23:14:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.287+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d819fa80 con 0x7f85d81033e0 2024-04-24T23:14:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.287+0000 7f85dc927640 1 -- 172.21.15.112:0/2021941230 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0046870 con 0x7f85d81033e0 2024-04-24T23:14:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.287+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81a0050 con 0x7f85d81033e0 2024-04-24T23:14:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.288+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8108af0 con 0x7f85d81033e0 2024-04-24T23:14:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.289+0000 7f85dc927640 1 -- 172.21.15.112:0/2021941230 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f85d0046a10 con 0x7f85d81033e0 2024-04-24T23:14:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.289+0000 7f85dc927640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f85c00a2b90 0x7f85c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.290+0000 7f85dc927640 1 -- 172.21.15.112:0/2021941230 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f85d002e030 con 0x7f85d81033e0 2024-04-24T23:14:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.290+0000 7f85d6ffd640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f85c00a2b90 0x7f85c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.290+0000 7f85d6ffd640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f85c00a2b90 0x7f85c00a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f85d8069c40 tx=0x7f85c8005c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.294+0000 7f85dc927640 1 -- 172.21.15.112:0/2021941230 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f85d004b060 con 0x7f85d81033e0 2024-04-24T23:14:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:15 smithi150 ceph-mon[32247]: pgmap v346: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:15 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/501260011' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:15 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2195251325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:15 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/289670889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:15 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1614284056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:16.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.506+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85d802d170 con 0x7f85d81033e0 2024-04-24T23:14:16.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.506+0000 7f85dc927640 1 -- 172.21.15.112:0/2021941230 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11947 (secure 0 0 0) 0x7f85d00bdbc0 con 0x7f85d81033e0 2024-04-24T23:14:16.508 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:16.508 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:12:55.876908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24482,24490,24425,24424]},"id":1}]} 2024-04-24T23:14:16.508 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-04-24T23:14:16.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f85c00a2b90 msgr2=0x7f85c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f85c00a2b90 0x7f85c00a5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f85d8069c40 tx=0x7f85c8005c10 comp rx=0 tx=0).stop 2024-04-24T23:14:16.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 msgr2=0x7f85d819f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 0x7f85d819f180 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009df0 tx=0x7f85d0003c10 comp rx=0 tx=0).stop 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 shutdown_connections 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f85c00a2b90 0x7f85c00a5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85d81033e0 0x7f85d819f180 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85d8102a20 0x7f85d8198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.508+0000 7f85dd929640 1 --2- 172.21.15.112:0/2021941230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85d80ffa50 0x7f85d8197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.509+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 >> 172.21.15.112:0/2021941230 conn(0x7f85d80fb380 msgr2=0x7f85d8105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.509+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 shutdown_connections 2024-04-24T23:14:16.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.509+0000 7f85dd929640 1 -- 172.21.15.112:0/2021941230 wait complete. 2024-04-24T23:14:16.521 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:16.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 -- 172.21.15.112:0/1646713750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f41006a0 msgr2=0x7ff1f4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/1646713750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f41006a0 0x7ff1f4100ac0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009900 tx=0x7ff1e802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:16.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 -- 172.21.15.112:0/1646713750 shutdown_connections 2024-04-24T23:14:16.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/1646713750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1f4102250 0x7ff1f4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/1646713750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f4101890 0x7ff1f4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/1646713750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f41006a0 0x7ff1f4100ac0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 -- 172.21.15.112:0/1646713750 >> 172.21.15.112:0/1646713750 conn(0x7ff1f40fbc30 msgr2=0x7ff1f40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 -- 172.21.15.112:0/1646713750 shutdown_connections 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.617+0000 7ff1fb15a640 1 -- 172.21.15.112:0/1646713750 wait complete. 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1fb15a640 1 Processor -- start 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1fb15a640 1 -- start start 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1fb15a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1f41006a0 0x7ff1f406a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1fb15a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4101890 0x7ff1f419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1fb15a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f4102250 0x7ff1f41a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1fb15a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f410c450 con 0x7ff1f4101890 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1fb15a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f410c2d0 con 0x7ff1f41006a0 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1fb15a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f410c5d0 con 0x7ff1f4102250 2024-04-24T23:14:16.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4101890 0x7ff1f419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4101890 0x7ff1f419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56344/0 (socket says 172.21.15.112:56344) 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 -- 172.21.15.112:0/3932672849 learned_addr learned my addr 172.21.15.112:0/3932672849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f8ef6640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1f41006a0 0x7ff1f406a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f96f7640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f4102250 0x7ff1f41a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 -- 172.21.15.112:0/3932672849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f4102250 msgr2=0x7ff1f41a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f4102250 0x7ff1f41a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 -- 172.21.15.112:0/3932672849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1f41006a0 msgr2=0x7ff1f406a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1f41006a0 0x7ff1f406a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 -- 172.21.15.112:0/3932672849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f41a1a30 con 0x7ff1f4101890 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f3fff640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4101890 0x7ff1f419ecf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000c970 tx=0x7ff1e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.618+0000 7ff1f1ffb640 1 -- 172.21.15.112:0/3932672849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0007bb0 con 0x7ff1f4101890 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.619+0000 7ff1f1ffb640 1 -- 172.21.15.112:0/3932672849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff1e0007d50 con 0x7ff1f4101890 2024-04-24T23:14:16.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.619+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f41a1cc0 con 0x7ff1f4101890 2024-04-24T23:14:16.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.619+0000 7ff1f1ffb640 1 -- 172.21.15.112:0/3932672849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000f4b0 con 0x7ff1f4101890 2024-04-24T23:14:16.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.619+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f41a21c0 con 0x7ff1f4101890 2024-04-24T23:14:16.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.620+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f41079c0 con 0x7ff1f4101890 2024-04-24T23:14:16.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.621+0000 7ff1f1ffb640 1 -- 172.21.15.112:0/3932672849 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff1e00046c0 con 0x7ff1f4101890 2024-04-24T23:14:16.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.621+0000 7ff1f1ffb640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff1e40a2bc0 0x7ff1e40a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.621+0000 7ff1f8ef6640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff1e40a2bc0 0x7ff1e40a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.621+0000 7ff1f1ffb640 1 -- 172.21.15.112:0/3932672849 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff1e00cf0e0 con 0x7ff1f4101890 2024-04-24T23:14:16.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.622+0000 7ff1f8ef6640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff1e40a2bc0 0x7ff1e40a5080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8031570 tx=0x7ff1e8031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:16.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.625+0000 7ff1f1ffb640 1 -- 172.21.15.112:0/3932672849 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff1e0014030 con 0x7ff1f4101890 2024-04-24T23:14:16.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.805+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1f4062340 con 0x7ff1f4101890 2024-04-24T23:14:16.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.806+0000 7ff1f1ffb640 1 -- 172.21.15.112:0/3932672849 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24714 (secure 0 0 0) 0x7ff1e009bce0 con 0x7ff1f4101890 2024-04-24T23:14:16.808 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:16.808 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":92,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:13:27.938988+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff1e40a2bc0 msgr2=0x7ff1e40a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff1e40a2bc0 0x7ff1e40a5080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8031570 tx=0x7ff1e8031640 comp rx=0 tx=0).stop 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4101890 msgr2=0x7ff1f419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4101890 0x7ff1f419ecf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000c970 tx=0x7ff1e000ce30 comp rx=0 tx=0).stop 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 shutdown_connections 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff1e40a2bc0 0x7ff1e40a5080 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff1f4102250 0x7ff1f41a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1f4101890 0x7ff1f419ecf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 --2- 172.21.15.112:0/3932672849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1f41006a0 0x7ff1f406a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.808+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 >> 172.21.15.112:0/3932672849 conn(0x7ff1f40fbc30 msgr2=0x7ff1f4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.809+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 shutdown_connections 2024-04-24T23:14:16.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.809+0000 7ff1fb15a640 1 -- 172.21.15.112:0/3932672849 wait complete. 2024-04-24T23:14:16.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:16.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 -- 172.21.15.112:0/3463584622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1018f0 msgr2=0x7f805c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 --2- 172.21.15.112:0/3463584622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1018f0 0x7f805c101d70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8050009900 tx=0x7f805002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 -- 172.21.15.112:0/3463584622 shutdown_connections 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 --2- 172.21.15.112:0/3463584622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805c1022b0 0x7f805c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 --2- 172.21.15.112:0/3463584622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c1018f0 0x7f805c101d70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 --2- 172.21.15.112:0/3463584622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f805c100700 0x7f805c100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 -- 172.21.15.112:0/3463584622 >> 172.21.15.112:0/3463584622 conn(0x7f805c0fbc90 msgr2=0x7f805c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 -- 172.21.15.112:0/3463584622 shutdown_connections 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.920+0000 7f8061037640 1 -- 172.21.15.112:0/3463584622 wait complete. 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f8061037640 1 Processor -- start 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f8061037640 1 -- start start 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f8061037640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c100700 0x7f805c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f8061037640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805c1018f0 0x7f805c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f8061037640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f805c1022b0 0x7f805c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f8061037640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f805c10c4c0 con 0x7f805c100700 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f8061037640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f805c10c340 con 0x7f805c1018f0 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f8061037640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f805c10c640 con 0x7f805c1022b0 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f805ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c100700 0x7f805c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f805ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c100700 0x7f805c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56350/0 (socket says 172.21.15.112:56350) 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f805ad9d640 1 -- 172.21.15.112:0/3650763324 learned_addr learned my addr 172.21.15.112:0/3650763324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.921+0000 7f805a59c640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805c1018f0 0x7f805c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f805ad9d640 1 -- 172.21.15.112:0/3650763324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f805c1022b0 msgr2=0x7f805c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f805b59e640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f805c1022b0 0x7f805c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f805ad9d640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f805c1022b0 0x7f805c19f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f805ad9d640 1 -- 172.21.15.112:0/3650763324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805c1018f0 msgr2=0x7f805c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f805ad9d640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805c1018f0 0x7f805c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f805ad9d640 1 -- 172.21.15.112:0/3650763324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c19f880 con 0x7f805c100700 2024-04-24T23:14:16.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f805ad9d640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c100700 0x7f805c197c70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f804c00b450 tx=0x7f804c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:16.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f8043fff640 1 -- 172.21.15.112:0/3650763324 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c015020 con 0x7f805c100700 2024-04-24T23:14:16.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c19fb70 con 0x7f805c100700 2024-04-24T23:14:16.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f8043fff640 1 -- 172.21.15.112:0/3650763324 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f804c011d90 con 0x7f805c100700 2024-04-24T23:14:16.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f8043fff640 1 -- 172.21.15.112:0/3650763324 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c00f340 con 0x7f805c100700 2024-04-24T23:14:16.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.922+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c1a00c0 con 0x7f805c100700 2024-04-24T23:14:16.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.923+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c1a0290 con 0x7f805c100700 2024-04-24T23:14:16.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.924+0000 7f8043fff640 1 -- 172.21.15.112:0/3650763324 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f804c018020 con 0x7f805c100700 2024-04-24T23:14:16.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.925+0000 7f8043fff640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80440a2b90 0x7f80440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:16.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.925+0000 7f805a59c640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80440a2b90 0x7f80440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:16.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.925+0000 7f8043fff640 1 -- 172.21.15.112:0/3650763324 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f804c0cf300 con 0x7f805c100700 2024-04-24T23:14:16.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.925+0000 7f805a59c640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80440a2b90 0x7f80440a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8050006010 tx=0x7f8050031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:16.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:16.928+0000 7f8043fff640 1 -- 172.21.15.112:0/3650763324 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f804c01f020 con 0x7f805c100700 2024-04-24T23:14:17.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.140+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f805c1a0460 con 0x7f805c100700 2024-04-24T23:14:17.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.141+0000 7f8043fff640 1 -- 172.21.15.112:0/3650763324 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11947 (secure 0 0 0) 0x7f804c00d040 con 0x7f805c100700 2024-04-24T23:14:17.143 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:17.143 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:12:55.876908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24482,24490,24425,24424]},"id":1}]} 2024-04-24T23:14:17.144 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-04-24T23:14:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80440a2b90 msgr2=0x7f80440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80440a2b90 0x7f80440a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8050006010 tx=0x7f8050031640 comp rx=0 tx=0).stop 2024-04-24T23:14:17.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c100700 msgr2=0x7f805c197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c100700 0x7f805c197c70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f804c00b450 tx=0x7f804c00b910 comp rx=0 tx=0).stop 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 shutdown_connections 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f80440a2b90 0x7f80440a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f805c1022b0 0x7f805c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f805c1018f0 0x7f805c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 --2- 172.21.15.112:0/3650763324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f805c100700 0x7f805c197c70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 >> 172.21.15.112:0/3650763324 conn(0x7f805c0fbc90 msgr2=0x7f805c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 shutdown_connections 2024-04-24T23:14:17.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.143+0000 7f8061037640 1 -- 172.21.15.112:0/3650763324 wait complete. 2024-04-24T23:14:17.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:16 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3348906453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:17.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:16 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2021941230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:17.155 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:16 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3932672849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:17.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T23:12:27.330443+0000', 'modified': '2024-04-24T23:12:55.876908+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24490, 'mds_2': 24482, 'mds_3': 24425, 'mds_4': 24424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6839/2592086011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2592086011}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2592086011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6841/3500837168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 3500837168}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 3500837168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.187:6833/1998066775', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1998066775}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 1998066775}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.150:6833/68455606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6832', 'nonce': 68455606}, {'type': 'v1', 'addr': '172.21.15.150:6833', 'nonce': 68455606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/2076270934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 2076270934}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 2076270934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/1908796622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 1908796622}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 1908796622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6837/35756805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 35756805}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 35756805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.150:6837/3592489674', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 3592489674}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 3592489674}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.187:6839/3343164580', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 3343164580}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 3343164580}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/748900426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 748900426}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 748900426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24524, 24482, 24490, 24425, 24424]} 2024-04-24T23:14:17.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-24T23:14:17.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-24T23:14:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:16 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3348906453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:16 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2021941230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:16 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3932672849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 -- 172.21.15.112:0/2450539861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8104b90 msgr2=0x7f0bd8106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 --2- 172.21.15.112:0/2450539861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8104b90 0x7f0bd8106f80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009900 tx=0x7f0bc402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 -- 172.21.15.112:0/2450539861 shutdown_connections 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 --2- 172.21.15.112:0/2450539861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bd81074c0 0x7f0bd81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 --2- 172.21.15.112:0/2450539861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8104b90 0x7f0bd8106f80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 --2- 172.21.15.112:0/2450539861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8102260 0x7f0bd8104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 -- 172.21.15.112:0/2450539861 >> 172.21.15.112:0/2450539861 conn(0x7f0bd80fbc30 msgr2=0x7f0bd80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 -- 172.21.15.112:0/2450539861 shutdown_connections 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 -- 172.21.15.112:0/2450539861 wait complete. 2024-04-24T23:14:17.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 Processor -- start 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.254+0000 7f0bdef16640 1 -- start start 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdef16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8102260 0x7f0bd8101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdef16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8104b90 0x7f0bd8100210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdef16640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bd81074c0 0x7f0bd8100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdef16640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd810d550 con 0x7f0bd8102260 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdef16640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd810d3d0 con 0x7f0bd81074c0 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdef16640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd810d6d0 con 0x7f0bd8104b90 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8102260 0x7f0bd8101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8102260 0x7f0bd8101bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56360/0 (socket says 172.21.15.112:56360) 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 -- 172.21.15.112:0/3579047186 learned_addr learned my addr 172.21.15.112:0/3579047186 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8104b90 0x7f0bd8100210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdd4b3640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bd81074c0 0x7f0bd8100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 -- 172.21.15.112:0/3579047186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8104b90 msgr2=0x7f0bd8100210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8104b90 0x7f0bd8100210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 -- 172.21.15.112:0/3579047186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bd81074c0 msgr2=0x7f0bd8100750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bd81074c0 0x7f0bd8100750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 -- 172.21.15.112:0/3579047186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd8100fe0 con 0x7f0bd8102260 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdd4b3640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bd81074c0 0x7f0bd8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8102260 0x7f0bd8101bd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c890 tx=0x7f0bcc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bd5ffb640 1 -- 172.21.15.112:0/3579047186 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004220 con 0x7f0bd8102260 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bdef16640 1 -- 172.21.15.112:0/3579047186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd81a1cf0 con 0x7f0bd8102260 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bd5ffb640 1 -- 172.21.15.112:0/3579047186 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0bcc0043c0 con 0x7f0bd8102260 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.255+0000 7f0bd5ffb640 1 -- 172.21.15.112:0/3579047186 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004b10 con 0x7f0bd8102260 2024-04-24T23:14:17.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.256+0000 7f0bdef16640 1 -- 172.21.15.112:0/3579047186 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd81a21c0 con 0x7f0bd8102260 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.257+0000 7f0bd5ffb640 1 -- 172.21.15.112:0/3579047186 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0bcc0051b0 con 0x7f0bd8102260 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0bd5ffb640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0bc80a2b90 0x7f0bc80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0be1c1b740 1 Processor -- start 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0be1c1b740 1 -- start start 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0bd5ffb640 1 -- 172.21.15.112:0/3579047186 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0bcc0cf500 con 0x7f0bd8102260 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0bc80a2b90 0x7f0bc80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 0x558d1d38f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d394930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d397430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d1d2f2f00 con 0x558d1d38fe30 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558d1d29b750 con 0x558d1d38f440 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558d1d2eeec0 con 0x558d1d394fa0 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d394930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d394930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56370/0 (socket says 172.21.15.112:56370) 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0bd7fff640 1 -- 172.21.15.112:0/3680385073 learned_addr learned my addr 172.21.15.112:0/3680385073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.258+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 0x558d1d38f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0bc80a2b90 0x7f0bc80a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4006010 tx=0x7f0bc4032d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bdd4b3640 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d397430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bd7fff640 1 -- 172.21.15.112:0/3680385073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 msgr2=0x558d1d397430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d397430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bd7fff640 1 -- 172.21.15.112:0/3680385073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 msgr2=0x558d1d38f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 0x558d1d38f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bd7fff640 1 -- 172.21.15.112:0/3680385073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1d0072d0 con 0x558d1d38fe30 2024-04-24T23:14:17.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d394930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40096c0 tx=0x7f0bc4044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a673b955555cafc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bc2ffd640 1 -- 172.21.15.112:0/3680385073 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc404d070 con 0x558d1d38fe30 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bc2ffd640 1 -- 172.21.15.112:0/3680385073 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0bc4005540 con 0x558d1d38fe30 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0bc2ffd640 1 -- 172.21.15.112:0/3680385073 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4035050 con 0x558d1d38fe30 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.259+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3680385073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 msgr2=0x558d1d394930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d394930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40096c0 tx=0x7f0bc4044000 comp rx=0 tx=0).stop 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3680385073 shutdown_connections 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d397430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d394930 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/3680385073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 0x558d1d38f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3680385073 >> 172.21.15.112:0/3680385073 conn(0x558d1d38aab0 msgr2=0x558d1d38aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3680385073 shutdown_connections 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3680385073 wait complete. 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 -- start start 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 Processor -- start 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d399800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 0x558d1d399d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d3a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d1d397ef0 con 0x558d1d38f440 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558d1d2aecf0 con 0x558d1d38fe30 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558d1d398070 con 0x558d1d394fa0 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0be1c1b740 10 client.0 _ll_register_callbacks cb 0x558d1d2ae6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0bdccb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d399800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0bdccb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d399800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56384/0 (socket says 172.21.15.112:56384) 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.260+0000 7f0bdccb2640 1 -- 172.21.15.112:0/1233294161 learned_addr learned my addr 172.21.15.112:0/1233294161 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bdd4b3640 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d3a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 0x558d1d399d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bdccb2640 1 -- 172.21.15.112:0/1233294161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 msgr2=0x558d1d3a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d3a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bdccb2640 1 -- 172.21.15.112:0/1233294161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 msgr2=0x558d1d399d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 0x558d1d399d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bdccb2640 1 -- 172.21.15.112:0/1233294161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1d0d16b0 con 0x558d1d38f440 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d399800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc09b700 tx=0x7f0bcc09ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bb63546c01bb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1233294161 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc01d040 con 0x558d1d38f440 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1233294161 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0bcc0cf240 con 0x558d1d38f440 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1233294161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc4005540 con 0x558d1d38f440 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1233294161 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc0a41e0 con 0x558d1d38f440 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 10 client.15231 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1233294161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d1d3a3dd0 con 0x558d1d38f440 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1233294161 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0bc4005540 con 0x558d1d38f440 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1233294161 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0bcc004cb0 con 0x558d1d38f440 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 10 client.15231 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 10 client.15231 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-24T23:14:17.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 20 client.15231 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558d1d3a63a0 0x558d1d3a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 4 client.15231 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.261+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1233294161 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d1d300440 con 0x558d1d3a63a0 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.262+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558d1d3a63a0 0x558d1d3a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.262+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558d1d3a63a0 0x558d1d3a7810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f8ff1848833a3f2 server_cookie=fbf86f0df6a818a4 in_seq=0 out_seq=0 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.262+0000 7f0bb7fff640 10 client.15231 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1233294161 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d1d300440 con 0x558d1d3a63a0 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0bb7fff640 10 client.15231 handle_command_reply: tid=0 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 client.15231 shutdown 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1233294161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 msgr2=0x558d1d399800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d399800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc09b700 tx=0x7f0bcc09ba00 comp rx=0 tx=0).stop 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1233294161 shutdown_connections 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558d1d3a63a0 0x558d1d3a7810 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d3a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 0x558d1d399d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0bb7fff640 0 client.15231 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:14:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1233294161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d399800 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1233294161 >> 172.21.15.112:0/1233294161 conn(0x558d1d38aab0 msgr2=0x558d1d389ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1233294161 shutdown_connections 2024-04-24T23:14:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1233294161 wait complete. 2024-04-24T23:14:17.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.263+0000 7f0be1c1b740 20 client.15231 trim_cache size 0 max 16384 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0be1c1b740 1 Processor -- start 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0be1c1b740 1 -- start start 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d394860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 0x558d1d3a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d399410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d1d3a5b50 con 0x558d1d38f440 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558d1d3a6220 con 0x558d1d38fe30 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558d1d3a68f0 con 0x558d1d394fa0 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d394860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d394860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56392/0 (socket says 172.21.15.112:56392) 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 -- 172.21.15.112:0/3327694180 learned_addr learned my addr 172.21.15.112:0/3327694180 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdd4b3640 1 --2- 172.21.15.112:0/3327694180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d399410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 -- 172.21.15.112:0/3327694180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 msgr2=0x558d1d399410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/3327694180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d399410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 -- 172.21.15.112:0/3327694180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 msgr2=0x558d1d3a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/3327694180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 0x558d1d3a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 -- 172.21.15.112:0/3327694180 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1d384c10 con 0x558d1d38f440 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.281+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/3327694180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d394860 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc016000 tx=0x7f0bcc099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f08d3025c89a47c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0bc27fc640 1 -- 172.21.15.112:0/3327694180 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc01d040 con 0x558d1d38f440 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0bc27fc640 1 -- 172.21.15.112:0/3327694180 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0bcc004cb0 con 0x558d1d38f440 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0bc27fc640 1 -- 172.21.15.112:0/3327694180 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0cf2a0 con 0x558d1d38f440 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3327694180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 msgr2=0x558d1d394860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/3327694180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d394860 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc016000 tx=0x7f0bcc099f10 comp rx=0 tx=0).stop 2024-04-24T23:14:17.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3327694180 shutdown_connections 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/3327694180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d399410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/3327694180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38fe30 0x558d1d3a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/3327694180 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38f440 0x558d1d394860 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3327694180 >> 172.21.15.112:0/3327694180 conn(0x558d1d38aab0 msgr2=0x558d1d3911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3327694180 shutdown_connections 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 -- 172.21.15.112:0/3327694180 wait complete. 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.282+0000 7f0be1c1b740 1 -- start start 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0be1c1b740 1 Processor -- start 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 0x558d1d39cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d39d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0be1c1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d3a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d1d3a7690 con 0x558d1d38fe30 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558d1d3a6fc0 con 0x558d1d38f440 2024-04-24T23:14:17.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0be1c1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558d1d3a7d60 con 0x558d1d394fa0 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0be1c1b740 10 client.0 _ll_register_callbacks cb 0x7f0bcc09b450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d39d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d39d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56402/0 (socket says 172.21.15.112:56402) 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1527627665 learned_addr learned my addr 172.21.15.112:0/1527627665 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bdd4b3640 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d3a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1527627665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 msgr2=0x558d1d3a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d3a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1527627665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 msgr2=0x558d1d39cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 0x558d1d39cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 -- 172.21.15.112:0/1527627665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1d3a07d0 con 0x558d1d38fe30 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bd7fff640 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d39d1c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40316a0 tx=0x7f0bc4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2fb0e443507a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.283+0000 7f0bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1527627665 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4058020 con 0x558d1d38fe30 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1527627665 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0bc4004240 con 0x558d1d38fe30 2024-04-24T23:14:17.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1527627665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bcc004cb0 con 0x558d1d38fe30 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1527627665 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0bc404d040 con 0x558d1d38fe30 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 10 client.15237 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1527627665 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d1d40cbb0 con 0x558d1d38fe30 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1527627665 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f0bcc004cb0 con 0x558d1d38fe30 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1527627665 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0bc40038c0 con 0x558d1d38fe30 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 10 client.15237 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 10 client.15237 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 20 client.15237 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558d1d411330 0x558d1d40d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 4 client.15237 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1527627665 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "status"}) -- 0x558d1d2fa790 con 0x558d1d411330 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.284+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558d1d411330 0x558d1d40d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0bdccb2640 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558d1d411330 0x558d1d40d770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=31a0c61e1ddb54e0 server_cookie=f58ba63ff0749c81 in_seq=0 out_seq=0 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0bb7fff640 10 client.15237 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0bb7fff640 1 -- 172.21.15.112:0/1527627665 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558d1d2fa790 con 0x558d1d411330 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0bb7fff640 10 client.15237 handle_command_reply: tid=0 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 client.15237 shutdown 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1527627665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 msgr2=0x558d1d39d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d39d1c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40316a0 tx=0x7f0bc4009880 comp rx=0 tx=0).stop 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1527627665 shutdown_connections 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558d1d411330 0x558d1d40d770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558d1d394fa0 0x558d1d3a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d1d38fe30 0x558d1d39d1c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0bb7fff640 0 client.15237 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 --2- 172.21.15.112:0/1527627665 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558d1d38f440 0x558d1d39cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1527627665 >> 172.21.15.112:0/1527627665 conn(0x558d1d38aab0 msgr2=0x558d1d38e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1527627665 shutdown_connections 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 1 -- 172.21.15.112:0/1527627665 wait complete. 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.285+0000 7f0be1c1b740 20 client.15237 trim_cache size 0 max 16384 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 4, 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24424, 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 82.413634135999999, 2024-04-24T23:14:17.294 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:17.295 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 92, 2024-04-24T23:14:17.295 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:17.295 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 147.35023218200001 2024-04-24T23:14:17.295 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:17.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 -- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0bc80a2b90 msgr2=0x7f0bc80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0bc80a2b90 0x7f0bc80a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4006010 tx=0x7f0bc4032d60 comp rx=0 tx=0).stop 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 -- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8102260 msgr2=0x7f0bd8101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8102260 0x7f0bd8101bd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c890 tx=0x7f0bcc00cd50 comp rx=0 tx=0).stop 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 -- 172.21.15.112:0/3579047186 shutdown_connections 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f0bc80a2b90 0x7f0bc80a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0bd81074c0 0x7f0bd8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bd8104b90 0x7f0bd8100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 --2- 172.21.15.112:0/3579047186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bd8102260 0x7f0bd8101bd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 -- 172.21.15.112:0/3579047186 >> 172.21.15.112:0/3579047186 conn(0x7f0bd80fbc30 msgr2=0x7f0bd8102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 -- 172.21.15.112:0/3579047186 shutdown_connections 2024-04-24T23:14:17.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.286+0000 7f0bdef16640 1 -- 172.21.15.112:0/3579047186 wait complete. 2024-04-24T23:14:17.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-24T23:14:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:16 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3348906453' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:16 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2021941230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:16 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3932672849' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:17.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 -- 172.21.15.112:0/2858514542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf01074e0 msgr2=0x7fdbf01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 --2- 172.21.15.112:0/2858514542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf01074e0 0x7fdbf01098d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009ca0 tx=0x7fdbe802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 -- 172.21.15.112:0/2858514542 shutdown_connections 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 --2- 172.21.15.112:0/2858514542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf01074e0 0x7fdbf01098d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 --2- 172.21.15.112:0/2858514542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf0104bb0 0x7fdbf0106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 --2- 172.21.15.112:0/2858514542 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbf0102280 0x7fdbf0104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 -- 172.21.15.112:0/2858514542 >> 172.21.15.112:0/2858514542 conn(0x7fdbf00fbc10 msgr2=0x7fdbf00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 -- 172.21.15.112:0/2858514542 shutdown_connections 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.397+0000 7fdbf629a640 1 -- 172.21.15.112:0/2858514542 wait complete. 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbf629a640 1 Processor -- start 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbf629a640 1 -- start start 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbf629a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf0102280 0x7fdbf0197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbf629a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbf0104bb0 0x7fdbf0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbf629a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf01074e0 0x7fdbf019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbf629a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf010d550 con 0x7fdbf0102280 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbf629a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf010d3d0 con 0x7fdbf0104bb0 2024-04-24T23:14:17.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbf629a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf010d6d0 con 0x7fdbf01074e0 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbeffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf0102280 0x7fdbf0197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbeffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf0102280 0x7fdbf0197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56412/0 (socket says 172.21.15.112:56412) 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbeffff640 1 -- 172.21.15.112:0/371553348 learned_addr learned my addr 172.21.15.112:0/371553348 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.398+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbf0104bb0 0x7fdbf0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbeffff640 1 -- 172.21.15.112:0/371553348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf01074e0 msgr2=0x7fdbf019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbeffff640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf01074e0 0x7fdbf019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbeffff640 1 -- 172.21.15.112:0/371553348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbf0104bb0 msgr2=0x7fdbf0198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbeffff640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbf0104bb0 0x7fdbf0198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbeffff640 1 -- 172.21.15.112:0/371553348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf019f700 con 0x7fdbf0102280 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbeffff640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf0102280 0x7fdbf0197ae0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000d950 tx=0x7fdbe000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbed7fa640 1 -- 172.21.15.112:0/371553348 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0017070 con 0x7fdbf0102280 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbf629a640 1 -- 172.21.15.112:0/371553348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf019f9f0 con 0x7fdbf0102280 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbed7fa640 1 -- 172.21.15.112:0/371553348 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdbe000de80 con 0x7fdbf0102280 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbed7fa640 1 -- 172.21.15.112:0/371553348 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000f620 con 0x7fdbf0102280 2024-04-24T23:14:17.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.399+0000 7fdbf629a640 1 -- 172.21.15.112:0/371553348 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf019ff90 con 0x7fdbf0102280 2024-04-24T23:14:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.401+0000 7fdbed7fa640 1 -- 172.21.15.112:0/371553348 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdbe0009c30 con 0x7fdbf0102280 2024-04-24T23:14:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf8fa5740 1 Processor -- start 2024-04-24T23:14:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf8fa5740 1 -- start start 2024-04-24T23:14:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321eeb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eebe30 0x55f321ef0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321ef3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f321e4ef00 con 0x55f321ef0fa0 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f321df7750 con 0x55f321eeb440 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f321e4aec0 con 0x55f321eebe30 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321ef3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbeffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eebe30 0x55f321ef0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321ef3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56420/0 (socket says 172.21.15.112:56420) 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 -- 172.21.15.112:0/4221885315 learned_addr learned my addr 172.21.15.112:0/4221885315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbed7fa640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdbd80a2b90 0x7fdbd80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf4837640 1 --2- 172.21.15.112:0/4221885315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321eeb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbf4837640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdbd80a2b90 0x7fdbd80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 -- 172.21.15.112:0/4221885315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eebe30 msgr2=0x55f321ef0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/4221885315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eebe30 0x55f321ef0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 -- 172.21.15.112:0/4221885315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 msgr2=0x55f321eeb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/4221885315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321eeb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 -- 172.21.15.112:0/4221885315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f321b632d0 con 0x55f321ef0fa0 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbed7fa640 1 -- 172.21.15.112:0/371553348 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdbe00cf120 con 0x7fdbf0102280 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.402+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/4221885315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321ef3430 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400ac70 tx=0x7fdbe400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69351e1097bed66e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbd67fc640 1 -- 172.21.15.112:0/4221885315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe401d070 con 0x55f321ef0fa0 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf4837640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdbd80a2b90 0x7fdbd80a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdbe802ade0 tx=0x7fdbe80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbd67fc640 1 -- 172.21.15.112:0/4221885315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdbe4018b40 con 0x55f321ef0fa0 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbd67fc640 1 -- 172.21.15.112:0/4221885315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4010d50 con 0x55f321ef0fa0 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/4221885315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 msgr2=0x55f321ef3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/4221885315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321ef3430 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400ac70 tx=0x7fdbe400f790 comp rx=0 tx=0).stop 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/4221885315 shutdown_connections 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/4221885315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321ef3430 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/4221885315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eebe30 0x55f321ef0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/4221885315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321eeb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/4221885315 >> 172.21.15.112:0/4221885315 conn(0x55f321ee6ab0 msgr2=0x55f321ee6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/4221885315 shutdown_connections 2024-04-24T23:14:17.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.403+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/4221885315 wait complete. 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 1 -- start start 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 1 Processor -- start 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eeb440 0x55f321ef57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eebe30 0x55f321ef5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321efcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f321ef3f50 con 0x55f321ef0fa0 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f321e0acf0 con 0x55f321eebe30 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f321ef40d0 con 0x55f321eeb440 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf8fa5740 10 client.0 _ll_register_callbacks cb 0x55f321e0a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf4837640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321efcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf4837640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321efcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56432/0 (socket says 172.21.15.112:56432) 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbf4837640 1 -- 172.21.15.112:0/953994669 learned_addr learned my addr 172.21.15.112:0/953994669 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.404+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eebe30 0x55f321ef5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbeffff640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eeb440 0x55f321ef57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf4837640 1 -- 172.21.15.112:0/953994669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eeb440 msgr2=0x55f321ef57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf4837640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eeb440 0x55f321ef57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf4837640 1 -- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eebe30 msgr2=0x55f321ef5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf4837640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eebe30 0x55f321ef5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf4837640 1 -- 172.21.15.112:0/953994669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f321c2d6b0 con 0x55f321ef0fa0 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eebe30 0x55f321ef5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbeffff640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eeb440 0x55f321ef57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf4837640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321efcd70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8035000 tx=0x7fdbe8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d43c130819e145 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/953994669 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe802fd10 con 0x55f321ef0fa0 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/953994669 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdbe80305c0 con 0x55f321ef0fa0 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/953994669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe4018b40 con 0x55f321ef0fa0 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/953994669 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdbe80098d0 con 0x55f321ef0fa0 2024-04-24T23:14:17.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf8fa5740 10 client.15249 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/953994669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f321effd50 con 0x55f321ef0fa0 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/953994669 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdbe4018b40 con 0x55f321ef0fa0 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/953994669 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdbe80507d0 con 0x55f321ef0fa0 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.405+0000 7fdbf8fa5740 10 client.15249 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.406+0000 7fdbf8fa5740 10 client.15249 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.406+0000 7fdbf8fa5740 20 client.15249 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.406+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55f321f02560 0x55f321f039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.406+0000 7fdbf8fa5740 4 client.15249 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.406+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/953994669 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f321e5c440 con 0x55f321f02560 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.406+0000 7fdbeffff640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55f321f02560 0x55f321f039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.406+0000 7fdbcf7fe640 10 client.15249 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.406+0000 7fdbeffff640 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55f321f02560 0x55f321f039d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ad430d3f6534883 server_cookie=c894f392b3b5be15 in_seq=0 out_seq=0 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/953994669 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f321e5c440 con 0x55f321f02560 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbcf7fe640 10 client.15249 handle_command_reply: tid=0 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 client.15249 shutdown 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/953994669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 msgr2=0x55f321efcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321efcd70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8035000 tx=0x7fdbe8005c50 comp rx=0 tx=0).stop 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/953994669 shutdown_connections 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55f321f02560 0x55f321f039d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321ef0fa0 0x55f321efcd70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eebe30 0x55f321ef5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbcf7fe640 0 client.15249 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/953994669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321eeb440 0x55f321ef57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/953994669 >> 172.21.15.112:0/953994669 conn(0x55f321ee6ab0 msgr2=0x55f321ee5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/953994669 shutdown_connections 2024-04-24T23:14:17.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/953994669 wait complete. 2024-04-24T23:14:17.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.407+0000 7fdbf8fa5740 20 client.15249 trim_cache size 0 max 16384 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.425+0000 7fdbf8fa5740 1 Processor -- start 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.425+0000 7fdbf8fa5740 1 -- start start 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.425+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321f01100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.425+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.425+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 0x55f321f01640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.425+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f321f01d10 con 0x55f321eebe30 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.425+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f321f023e0 con 0x55f321eeb440 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.425+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f321f02ab0 con 0x55f321ef0fa0 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56440/0 (socket says 172.21.15.112:56440) 2024-04-24T23:14:17.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 -- 172.21.15.112:0/3620734435 learned_addr learned my addr 172.21.15.112:0/3620734435 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 -- 172.21.15.112:0/3620734435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 msgr2=0x55f321f01640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/3620734435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 0x55f321f01640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbeffff640 1 --2- 172.21.15.112:0/3620734435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321f01100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 -- 172.21.15.112:0/3620734435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 msgr2=0x55f321f01100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/3620734435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321f01100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 -- 172.21.15.112:0/3620734435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f321eea9d0 con 0x55f321eebe30 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/3620734435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efcd30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400ab90 tx=0x7fdbe400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677d451f8496cc01 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbd5ffb640 1 -- 172.21.15.112:0/3620734435 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4027070 con 0x55f321eebe30 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbd5ffb640 1 -- 172.21.15.112:0/3620734435 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdbe401e040 con 0x55f321eebe30 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.426+0000 7fdbd5ffb640 1 -- 172.21.15.112:0/3620734435 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe401d070 con 0x55f321eebe30 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/3620734435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 msgr2=0x55f321efcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/3620734435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efcd30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400ab90 tx=0x7fdbe400a650 comp rx=0 tx=0).stop 2024-04-24T23:14:17.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/3620734435 shutdown_connections 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/3620734435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 0x55f321f01640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/3620734435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efcd30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/3620734435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321f01100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/3620734435 >> 172.21.15.112:0/3620734435 conn(0x55f321ee6ab0 msgr2=0x55f321ee6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/3620734435 shutdown_connections 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/3620734435 wait complete. 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- start start 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 Processor -- start 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321efaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 0x55f321efba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f321f03850 con 0x55f321eebe30 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f321f03180 con 0x55f321eeb440 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.427+0000 7fdbf8fa5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f321f03f20 con 0x55f321ef0fa0 2024-04-24T23:14:17.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbf8fa5740 10 client.0 _ll_register_callbacks cb 0x7fdbe80318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56442/0 (socket says 172.21.15.112:56442) 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 -- 172.21.15.112:0/635620844 learned_addr learned my addr 172.21.15.112:0/635620844 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbeffff640 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321efaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 -- 172.21.15.112:0/635620844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 msgr2=0x55f321efba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbf4837640 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 0x55f321efba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 0x55f321efba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 -- 172.21.15.112:0/635620844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 msgr2=0x55f321efaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321efaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 -- 172.21.15.112:0/635620844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f321efc260 con 0x55f321eebe30 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbef7fe640 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efb4b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4005890 tx=0x7fdbe4010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7431e86ea05f7adf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/635620844 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4032070 con 0x55f321eebe30 2024-04-24T23:14:17.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/635620844 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdbe4018950 con 0x55f321eebe30 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/635620844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe401e040 con 0x55f321eebe30 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/635620844 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdbe403e050 con 0x55f321eebe30 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbf8fa5740 10 client.15255 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/635620844 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f321f68800 con 0x55f321eebe30 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.428+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/635620844 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fdbe401e040 con 0x55f321eebe30 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/635620844 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdbe40233f0 con 0x55f321eebe30 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbf8fa5740 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbf8fa5740 10 client.15255 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbf8fa5740 20 client.15255 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55f321f6ccb0 0x55f321f6b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbf8fa5740 4 client.15255 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/635620844 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "status"}) -- 0x55f321e56790 con 0x55f321f6ccb0 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbeffff640 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55f321f6ccb0 0x55f321f6b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbeffff640 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55f321f6ccb0 0x55f321f6b9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e1ba3e360daf5749 server_cookie=22c84fed8ece4a21 in_seq=0 out_seq=0 2024-04-24T23:14:17.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.429+0000 7fdbcf7fe640 10 client.15255 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbcf7fe640 1 -- 172.21.15.112:0/635620844 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f321e56790 con 0x55f321f6ccb0 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbcf7fe640 10 client.15255 handle_command_reply: tid=0 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 client.15255 shutdown 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/635620844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 msgr2=0x55f321efb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efb4b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4005890 tx=0x7fdbe4010900 comp rx=0 tx=0).stop 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/635620844 shutdown_connections 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55f321f6ccb0 0x55f321f6b9c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f321ef0fa0 0x55f321efba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f321eebe30 0x55f321efb4b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbcf7fe640 0 client.15255 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 --2- 172.21.15.112:0/635620844 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f321eeb440 0x55f321efaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/635620844 >> 172.21.15.112:0/635620844 conn(0x55f321ee6ab0 msgr2=0x55f321ee6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/635620844 shutdown_connections 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 1 -- 172.21.15.112:0/635620844 wait complete. 2024-04-24T23:14:17.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf8fa5740 20 client.15255 trim_cache size 0 max 16384 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 3, 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24425, 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 83.565899349000006, 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:17.439 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 92, 2024-04-24T23:14:17.440 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:17.440 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 144.56905007099999 2024-04-24T23:14:17.440 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:17.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf629a640 1 -- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdbd80a2b90 msgr2=0x7fdbd80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf629a640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdbd80a2b90 0x7fdbd80a5050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdbe802ade0 tx=0x7fdbe80095c0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf629a640 1 -- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf0102280 msgr2=0x7fdbf0197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.430+0000 7fdbf629a640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf0102280 0x7fdbf0197ae0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000d950 tx=0x7fdbe000dd50 comp rx=0 tx=0).stop 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.431+0000 7fdbf629a640 1 -- 172.21.15.112:0/371553348 shutdown_connections 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.431+0000 7fdbf629a640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fdbd80a2b90 0x7fdbd80a5050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.431+0000 7fdbf629a640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbf01074e0 0x7fdbf019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.431+0000 7fdbf629a640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbf0104bb0 0x7fdbf0198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.431+0000 7fdbf629a640 1 --2- 172.21.15.112:0/371553348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdbf0102280 0x7fdbf0197ae0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.431+0000 7fdbf629a640 1 -- 172.21.15.112:0/371553348 >> 172.21.15.112:0/371553348 conn(0x7fdbf00fbc10 msgr2=0x7fdbf0102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.431+0000 7fdbf629a640 1 -- 172.21.15.112:0/371553348 shutdown_connections 2024-04-24T23:14:17.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.431+0000 7fdbf629a640 1 -- 172.21.15.112:0/371553348 wait complete. 2024-04-24T23:14:17.443 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-24T23:14:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 -- 172.21.15.112:0/2534096461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf001006a0 msgr2=0x7faf00100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 --2- 172.21.15.112:0/2534096461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf001006a0 0x7faf00100ac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7faefc009900 tx=0x7faefc02a410 comp rx=0 tx=0).stop 2024-04-24T23:14:17.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 -- 172.21.15.112:0/2534096461 shutdown_connections 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 --2- 172.21.15.112:0/2534096461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf00102250 0x7faf00108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 --2- 172.21.15.112:0/2534096461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf00101890 0x7faf00101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 --2- 172.21.15.112:0/2534096461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf001006a0 0x7faf00100ac0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 -- 172.21.15.112:0/2534096461 >> 172.21.15.112:0/2534096461 conn(0x7faf000fbc30 msgr2=0x7faf000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 -- 172.21.15.112:0/2534096461 shutdown_connections 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.540+0000 7faf08763640 1 -- 172.21.15.112:0/2534096461 wait complete. 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 Processor -- start 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 -- start start 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf001006a0 0x7faf00197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf00101890 0x7faf00198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf00102250 0x7faf0019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf0010c450 con 0x7faf00101890 2024-04-24T23:14:17.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faf0010c2d0 con 0x7faf00102250 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faf0010c5d0 con 0x7faf001006a0 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf00101890 0x7faf00198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf00101890 0x7faf00198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56452/0 (socket says 172.21.15.112:56452) 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 -- 172.21.15.112:0/3335296841 learned_addr learned my addr 172.21.15.112:0/3335296841 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf064ff640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf001006a0 0x7faf00197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf06d00640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf00102250 0x7faf0019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 -- 172.21.15.112:0/3335296841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf001006a0 msgr2=0x7faf00197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf001006a0 0x7faf00197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 -- 172.21.15.112:0/3335296841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf00102250 msgr2=0x7faf0019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf00102250 0x7faf0019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 -- 172.21.15.112:0/3335296841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0019f7e0 con 0x7faf00101890 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf06d00640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf00102250 0x7faf0019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf05cfe640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf00101890 0x7faf00198100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faef000d950 tx=0x7faef000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faef77fe640 1 -- 172.21.15.112:0/3335296841 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0017070 con 0x7faf00101890 2024-04-24T23:14:17.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faef77fe640 1 -- 172.21.15.112:0/3335296841 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faef0009c30 con 0x7faf00101890 2024-04-24T23:14:17.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faf08763640 1 -- 172.21.15.112:0/3335296841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0019fad0 con 0x7faf00101890 2024-04-24T23:14:17.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.541+0000 7faef77fe640 1 -- 172.21.15.112:0/3335296841 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef000f740 con 0x7faf00101890 2024-04-24T23:14:17.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.542+0000 7faf08763640 1 -- 172.21.15.112:0/3335296841 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf001a0020 con 0x7faf00101890 2024-04-24T23:14:17.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faef77fe640 1 -- 172.21.15.112:0/3335296841 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7faef0010a80 con 0x7faf00101890 2024-04-24T23:14:17.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf0b47c740 1 Processor -- start 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf0b47c740 1 -- start start 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840440 0x562090840860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 0x562090845930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090845fa0 0x562090848430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf0b47c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5620907a3f00 con 0x562090845fa0 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf0b47c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56209074c750 con 0x562090840e30 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf0b47c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56209079fec0 con 0x562090840440 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf05cfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090845fa0 0x562090848430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf05cfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090845fa0 0x562090848430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56458/0 (socket says 172.21.15.112:56458) 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf05cfe640 1 -- 172.21.15.112:0/54887622 learned_addr learned my addr 172.21.15.112:0/54887622 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf064ff640 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 0x562090845930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faf06d00640 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840440 0x562090840860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.544+0000 7faef77fe640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faeec0a2b90 0x7faeec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf05cfe640 1 -- 172.21.15.112:0/54887622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840440 msgr2=0x562090840860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf05cfe640 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840440 0x562090840860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf05cfe640 1 -- 172.21.15.112:0/54887622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 msgr2=0x562090845930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf05cfe640 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 0x562090845930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf06d00640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faeec0a2b90 0x7faeec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf05cfe640 1 -- 172.21.15.112:0/54887622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620904b82d0 con 0x562090845fa0 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf064ff640 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 0x562090845930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faef77fe640 1 -- 172.21.15.112:0/3335296841 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faef00cf030 con 0x7faf00101890 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf05cfe640 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090845fa0 0x562090848430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faef0097f00 tx=0x7faef0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368d0669ef0cd3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faf06d00640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faeec0a2b90 0x7faeec0a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x562090841690 tx=0x7faef8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faef4ff9640 1 -- 172.21.15.112:0/54887622 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef00bb070 con 0x562090845fa0 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faef4ff9640 1 -- 172.21.15.112:0/54887622 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faef0009de0 con 0x562090845fa0 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.545+0000 7faef4ff9640 1 -- 172.21.15.112:0/54887622 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0020070 con 0x562090845fa0 2024-04-24T23:14:17.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- 172.21.15.112:0/54887622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090845fa0 msgr2=0x562090848430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090845fa0 0x562090848430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faef0097f00 tx=0x7faef0022000 comp rx=0 tx=0).stop 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- 172.21.15.112:0/54887622 shutdown_connections 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090845fa0 0x562090848430 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 0x562090845930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 --2- 172.21.15.112:0/54887622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840440 0x562090840860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- 172.21.15.112:0/54887622 >> 172.21.15.112:0/54887622 conn(0x56209083bab0 msgr2=0x56209083beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- 172.21.15.112:0/54887622 shutdown_connections 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- 172.21.15.112:0/54887622 wait complete. 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- start start 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 Processor -- start 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209084a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840e30 0x56209084acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090845fa0 0x562090851c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562090848e90 con 0x562090840440 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56209075fcf0 con 0x562090845fa0 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562090849010 con 0x562090840e30 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.546+0000 7faf0b47c740 10 client.0 _ll_register_callbacks cb 0x56209075f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209084a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209084a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56474/0 (socket says 172.21.15.112:56474) 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 -- 172.21.15.112:0/3598170980 learned_addr learned my addr 172.21.15.112:0/3598170980 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf06d00640 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090845fa0 0x562090851c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 -- 172.21.15.112:0/3598170980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840e30 msgr2=0x56209084acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840e30 0x56209084acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 -- 172.21.15.112:0/3598170980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090845fa0 msgr2=0x562090851c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090845fa0 0x562090851c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 -- 172.21.15.112:0/3598170980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620905826b0 con 0x562090840440 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf064ff640 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209084a770 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faf001016f0 tx=0x7faefc031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54c5aa93a7a5cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faee9ffb640 1 -- 172.21.15.112:0/3598170980 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc035040 con 0x562090840440 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faee9ffb640 1 -- 172.21.15.112:0/3598170980 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faefc004380 con 0x562090840440 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf0b47c740 1 -- 172.21.15.112:0/3598170980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef0009de0 con 0x562090840440 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faee9ffb640 1 -- 172.21.15.112:0/3598170980 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faefc005020 con 0x562090840440 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf0b47c740 10 client.15267 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faf0b47c740 1 -- 172.21.15.112:0/3598170980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562090854d20 con 0x562090840440 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.547+0000 7faee9ffb640 1 -- 172.21.15.112:0/3598170980 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faef0009de0 con 0x562090840440 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faee9ffb640 1 -- 172.21.15.112:0/3598170980 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7faefc0035f0 con 0x562090840440 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faf0b47c740 10 client.15267 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faf0b47c740 10 client.15267 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faf0b47c740 20 client.15267 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faf0b47c740 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x562090857440 0x5620908588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faf0b47c740 4 client.15267 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:17.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faf0b47c740 1 -- 172.21.15.112:0/3598170980 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620907b1440 con 0x562090857440 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faf05cfe640 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x562090857440 0x5620908588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faee9ffb640 10 client.15267 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.548+0000 7faf05cfe640 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x562090857440 0x5620908588b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d8f60c7e9fabef9 server_cookie=e2e08f617e4c118a in_seq=0 out_seq=0 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faee9ffb640 1 -- 172.21.15.112:0/3598170980 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620907b1440 con 0x562090857440 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faee9ffb640 10 client.15267 handle_command_reply: tid=0 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 client.15267 shutdown 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 -- 172.21.15.112:0/3598170980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 msgr2=0x56209084a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209084a770 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faf001016f0 tx=0x7faefc031ab0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 -- 172.21.15.112:0/3598170980 shutdown_connections 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x562090857440 0x5620908588b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090845fa0 0x562090851c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090840e30 0x56209084acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 --2- 172.21.15.112:0/3598170980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209084a770 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faee9ffb640 0 client.15267 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:14:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 -- 172.21.15.112:0/3598170980 >> 172.21.15.112:0/3598170980 conn(0x56209083bab0 msgr2=0x56209083aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 -- 172.21.15.112:0/3598170980 shutdown_connections 2024-04-24T23:14:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 1 -- 172.21.15.112:0/3598170980 wait complete. 2024-04-24T23:14:17.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.549+0000 7faf0b47c740 20 client.15267 trim_cache size 0 max 16384 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf0b47c740 1 Processor -- start 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf0b47c740 1 -- start start 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209083be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 0x562090851c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 0x56209083e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf0b47c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562090856bf0 con 0x562090840440 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf0b47c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5620908572c0 con 0x562090840e30 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf0b47c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562090857990 con 0x562090845fa0 2024-04-24T23:14:17.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf064ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209083be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf064ff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209083be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56488/0 (socket says 172.21.15.112:56488) 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf064ff640 1 -- 172.21.15.112:0/1311845971 learned_addr learned my addr 172.21.15.112:0/1311845971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf06d00640 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 0x56209083e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf064ff640 1 -- 172.21.15.112:0/1311845971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 msgr2=0x56209083e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf064ff640 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 0x56209083e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf064ff640 1 -- 172.21.15.112:0/1311845971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 msgr2=0x562090851c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf064ff640 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 0x562090851c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.567+0000 7faf064ff640 1 -- 172.21.15.112:0/1311845971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562090841210 con 0x562090840440 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf06d00640 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 0x56209083e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf064ff640 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209083be90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faefc003a60 tx=0x7faefc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa9151bbab05ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faeebfff640 1 -- 172.21.15.112:0/1311845971 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc035030 con 0x562090840440 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faeebfff640 1 -- 172.21.15.112:0/1311845971 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faefc003c50 con 0x562090840440 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faeebfff640 1 -- 172.21.15.112:0/1311845971 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc0049e0 con 0x562090840440 2024-04-24T23:14:17.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 -- 172.21.15.112:0/1311845971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 msgr2=0x56209083be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209083be90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faefc003a60 tx=0x7faefc005b50 comp rx=0 tx=0).stop 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 -- 172.21.15.112:0/1311845971 shutdown_connections 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 0x56209083e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840e30 0x562090851c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 --2- 172.21.15.112:0/1311845971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840440 0x56209083be90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 -- 172.21.15.112:0/1311845971 >> 172.21.15.112:0/1311845971 conn(0x56209083bab0 msgr2=0x5620908416c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 -- 172.21.15.112:0/1311845971 shutdown_connections 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.568+0000 7faf0b47c740 1 -- 172.21.15.112:0/1311845971 wait complete. 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 1 -- start start 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 1 Processor -- start 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840440 0x56209084d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840e30 0x56209084dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 0x562090850d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562090858730 con 0x562090840e30 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562090858060 con 0x562090840440 2024-04-24T23:14:17.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562090858e00 con 0x562090845fa0 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf0b47c740 10 client.0 _ll_register_callbacks cb 0x7faefc031740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840e30 0x56209084dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840e30 0x56209084dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56496/0 (socket says 172.21.15.112:56496) 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 -- 172.21.15.112:0/640967953 learned_addr learned my addr 172.21.15.112:0/640967953 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf064ff640 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840440 0x56209084d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 -- 172.21.15.112:0/640967953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 msgr2=0x562090850d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 0x562090850d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 -- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840440 msgr2=0x56209084d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840440 0x56209084d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 -- 172.21.15.112:0/640967953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562090851300 con 0x562090840e30 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf064ff640 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840440 0x56209084d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faf05cfe640 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840e30 0x56209084dcf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faef00ccb40 tx=0x7faef0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b5b728eb445ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.569+0000 7faee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faee9ffb640 1 -- 172.21.15.112:0/640967953 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0084070 con 0x562090840e30 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faee9ffb640 1 -- 172.21.15.112:0/640967953 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faef00af690 con 0x562090840e30 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 1 -- 172.21.15.112:0/640967953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faefc003c50 con 0x562090840e30 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faee9ffb640 1 -- 172.21.15.112:0/640967953 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faef0020040 con 0x562090840e30 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 1 -- 172.21.15.112:0/640967953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5620908bd7e0 con 0x562090840e30 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faee9ffb640 1 -- 172.21.15.112:0/640967953 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faefc003c50 con 0x562090840e30 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faee9ffb640 1 -- 172.21.15.112:0/640967953 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7faef0097ba0 con 0x562090840e30 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 10 client.15273 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 20 client.15273 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5620908c2030 0x5620908c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 4 client.15273 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf0b47c740 1 -- 172.21.15.112:0/640967953 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "status"}) -- 0x5620908491a0 con 0x5620908c2030 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.570+0000 7faf064ff640 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5620908c2030 0x5620908c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faee9ffb640 10 client.15273 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf064ff640 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5620908c2030 0x5620908c0d40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b29cfcabf6d5c62 server_cookie=bec1bd4347dd10df in_seq=0 out_seq=0 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faee9ffb640 1 -- 172.21.15.112:0/640967953 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5620908491a0 con 0x5620908c2030 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faee9ffb640 10 client.15273 handle_command_reply: tid=0 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 client.15273 shutdown 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 -- 172.21.15.112:0/640967953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840e30 msgr2=0x56209084dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840e30 0x56209084dcf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faef00ccb40 tx=0x7faef0000bc0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 -- 172.21.15.112:0/640967953 shutdown_connections 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5620908c2030 0x5620908c0d40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562090845fa0 0x562090850d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562090840e30 0x56209084dcf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faee9ffb640 0 client.15273 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 --2- 172.21.15.112:0/640967953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562090840440 0x56209084d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 -- 172.21.15.112:0/640967953 >> 172.21.15.112:0/640967953 conn(0x56209083bab0 msgr2=0x56209083be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 -- 172.21.15.112:0/640967953 shutdown_connections 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 1 -- 172.21.15.112:0/640967953 wait complete. 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.571+0000 7faf0b47c740 20 client.15273 trim_cache size 0 max 16384 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24482, 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 84.682304525000006, 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:17.580 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 92, 2024-04-24T23:14:17.581 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:17.581 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 127.623251916 2024-04-24T23:14:17.581 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 -- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faeec0a2b90 msgr2=0x7faeec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faeec0a2b90 0x7faeec0a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x562090841690 tx=0x7faef8004ab0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 -- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf00101890 msgr2=0x7faf00198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf00101890 0x7faf00198100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faef000d950 tx=0x7faef000de10 comp rx=0 tx=0).stop 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 -- 172.21.15.112:0/3335296841 shutdown_connections 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faeec0a2b90 0x7faeec0a5050 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf00102250 0x7faf0019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf00101890 0x7faf00198100 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 --2- 172.21.15.112:0/3335296841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faf001006a0 0x7faf00197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 -- 172.21.15.112:0/3335296841 >> 172.21.15.112:0/3335296841 conn(0x7faf000fbc30 msgr2=0x7faf00069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 -- 172.21.15.112:0/3335296841 shutdown_connections 2024-04-24T23:14:17.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.572+0000 7faf08763640 1 -- 172.21.15.112:0/3335296841 wait complete. 2024-04-24T23:14:17.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:14:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3147188360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0101870 msgr2=0x7fe5c0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3147188360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0101870 0x7fe5c0101cf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009900 tx=0x7fe5a802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3147188360 shutdown_connections 2024-04-24T23:14:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3147188360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5c0102230 0x7fe5c0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3147188360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0101870 0x7fe5c0101cf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3147188360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5c0100680 0x7fe5c0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3147188360 >> 172.21.15.112:0/3147188360 conn(0x7fe5c00fbbf0 msgr2=0x7fe5c00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3147188360 shutdown_connections 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.681+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3147188360 wait complete. 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5c5a12640 1 Processor -- start 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5c5a12640 1 -- start start 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5c5a12640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5c0100680 0x7fe5c0069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5c5a12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5c0101870 0x7fe5c019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5c5a12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0102230 0x7fe5c019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5c5a12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c010c430 con 0x7fe5c0102230 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5c5a12640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c010c2b0 con 0x7fe5c0100680 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5c5a12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c010c5b0 con 0x7fe5c0101870 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0102230 0x7fe5c019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5bf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0102230 0x7fe5c019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56512/0 (socket says 172.21.15.112:56512) 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5bf7fe640 1 -- 172.21.15.112:0/3381500298 learned_addr learned my addr 172.21.15.112:0/3381500298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5c0101870 0x7fe5c019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.682+0000 7fe5beffd640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5c0100680 0x7fe5c0069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5bf7fe640 1 -- 172.21.15.112:0/3381500298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5c0101870 msgr2=0x7fe5c019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5bf7fe640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5c0101870 0x7fe5c019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5bf7fe640 1 -- 172.21.15.112:0/3381500298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5c0100680 msgr2=0x7fe5c0069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5bf7fe640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5c0100680 0x7fe5c0069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5bf7fe640 1 -- 172.21.15.112:0/3381500298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c019f7d0 con 0x7fe5c0102230 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5c0101870 0x7fe5c019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5beffd640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5c0100680 0x7fe5c0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5bf7fe640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0102230 0x7fe5c019eec0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c970 tx=0x7fe5b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5c4a10640 1 -- 172.21.15.112:0/3381500298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8007bb0 con 0x7fe5c0102230 2024-04-24T23:14:17.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5c4a10640 1 -- 172.21.15.112:0/3381500298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe5b8007d50 con 0x7fe5c0102230 2024-04-24T23:14:17.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3381500298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c019fa60 con 0x7fe5c0102230 2024-04-24T23:14:17.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5c4a10640 1 -- 172.21.15.112:0/3381500298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80188a0 con 0x7fe5c0102230 2024-04-24T23:14:17.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.683+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3381500298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c019ff60 con 0x7fe5c0102230 2024-04-24T23:14:17.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c4a10640 1 -- 172.21.15.112:0/3381500298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe5b800f450 con 0x7fe5c0102230 2024-04-24T23:14:17.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c87f8740 1 Processor -- start 2024-04-24T23:14:17.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c87f8740 1 -- start start 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592440 0x55f490592860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592e30 0x55f490597930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f49059a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4904f5f00 con 0x55f490592e30 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f49049e750 con 0x55f490592440 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f4904f1ec0 con 0x55f490597fa0 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5beffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592e30 0x55f490597930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5beffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592e30 0x55f490597930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56520/0 (socket says 172.21.15.112:56520) 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5beffd640 1 -- 172.21.15.112:0/2770397383 learned_addr learned my addr 172.21.15.112:0/2770397383 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c4a10640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5ac0a2b90 0x7fe5ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5bf7fe640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5ac0a2b90 0x7fe5ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592440 0x55f490592860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5beffd640 1 -- 172.21.15.112:0/2770397383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 msgr2=0x55f49059a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5beffd640 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f49059a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5beffd640 1 -- 172.21.15.112:0/2770397383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592440 msgr2=0x55f490592860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5beffd640 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592440 0x55f490592860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5beffd640 1 -- 172.21.15.112:0/2770397383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49020a2d0 con 0x55f490592e30 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592440 0x55f490592860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.686+0000 7fe5c4a10640 1 -- 172.21.15.112:0/3381500298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe5b80cf5a0 con 0x7fe5c0102230 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5beffd640 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592e30 0x55f490597930 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009f10 tx=0x7fe5b000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaa6ee1a0042ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5b5ffb640 1 -- 172.21.15.112:0/2770397383 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b001a680 con 0x55f490592e30 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5bf7fe640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5ac0a2b90 0x7fe5ac0a5050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe5b809ba30 tx=0x7fe5b806eb60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5b5ffb640 1 -- 172.21.15.112:0/2770397383 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe5b0011940 con 0x55f490592e30 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5b5ffb640 1 -- 172.21.15.112:0/2770397383 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0010a40 con 0x55f490592e30 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5c87f8740 1 -- 172.21.15.112:0/2770397383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592e30 msgr2=0x55f490597930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592e30 0x55f490597930 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009f10 tx=0x7fe5b000f730 comp rx=0 tx=0).stop 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5c87f8740 1 -- 172.21.15.112:0/2770397383 shutdown_connections 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f49059a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592e30 0x55f490597930 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/2770397383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592440 0x55f490592860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5c87f8740 1 -- 172.21.15.112:0/2770397383 >> 172.21.15.112:0/2770397383 conn(0x55f49058dab0 msgr2=0x55f49058deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.687+0000 7fe5c87f8740 1 -- 172.21.15.112:0/2770397383 shutdown_connections 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 -- 172.21.15.112:0/2770397383 wait complete. 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 -- start start 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 Processor -- start 2024-04-24T23:14:17.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49059c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f49059ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f4905a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f49059af50 con 0x55f490592440 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f4904b1cf0 con 0x55f490592e30 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f49059b0d0 con 0x55f490597fa0 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5c87f8740 10 client.0 _ll_register_callbacks cb 0x55f4904b16e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49059c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49059c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56528/0 (socket says 172.21.15.112:56528) 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1838057101 learned_addr learned my addr 172.21.15.112:0/1838057101 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.688+0000 7fe5beffd640 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f49059ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1838057101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 msgr2=0x55f4905a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f4905a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1838057101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 msgr2=0x55f49059ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f49059ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1838057101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4902d46b0 con 0x55f490592440 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49059c920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8005ab0 tx=0x7fe5a80319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9906813292a285 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe59effd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe59effd640 1 -- 172.21.15.112:0/1838057101 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8040070 con 0x55f490592440 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe59effd640 1 -- 172.21.15.112:0/1838057101 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe5a8005050 con 0x55f490592440 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1838057101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0011940 con 0x55f490592440 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe59effd640 1 -- 172.21.15.112:0/1838057101 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5a804c050 con 0x55f490592440 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 10 client.15285 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1838057101 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f4905a6eb0 con 0x55f490592440 2024-04-24T23:14:17.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe59effd640 1 -- 172.21.15.112:0/1838057101 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe5b0011940 con 0x55f490592440 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe59effd640 1 -- 172.21.15.112:0/1838057101 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe5a80035f0 con 0x55f490592440 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 10 client.15285 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 20 client.15285 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f4905a9460 0x55f4905aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 4 client.15285 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.689+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1838057101 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f490503440 con 0x55f4905a9460 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.690+0000 7fe5beffd640 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f4905a9460 0x55f4905aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.690+0000 7fe59effd640 10 client.15285 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.690+0000 7fe5beffd640 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f4905a9460 0x55f4905aa8d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60c599a896c14fb7 server_cookie=b32d54787d56d950 in_seq=0 out_seq=0 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe59effd640 1 -- 172.21.15.112:0/1838057101 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f490503440 con 0x55f4905a9460 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe59effd640 10 client.15285 handle_command_reply: tid=0 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 client.15285 shutdown 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1838057101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 msgr2=0x55f49059c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49059c920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8005ab0 tx=0x7fe5a80319e0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1838057101 shutdown_connections 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f4905a9460 0x55f4905aa8d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f4905a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f49059ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1838057101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49059c920 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe59effd640 0 client.15285 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:14:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1838057101 >> 172.21.15.112:0/1838057101 conn(0x55f49058dab0 msgr2=0x55f49058cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1838057101 shutdown_connections 2024-04-24T23:14:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1838057101 wait complete. 2024-04-24T23:14:17.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.691+0000 7fe5c87f8740 20 client.15285 trim_cache size 0 max 16384 2024-04-24T23:14:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5c87f8740 1 Processor -- start 2024-04-24T23:14:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5c87f8740 1 -- start start 2024-04-24T23:14:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49058d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f4905a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f490588690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4905a8c10 con 0x55f490592440 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f4905a92e0 con 0x55f490592e30 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f4905a99b0 con 0x55f490597fa0 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49058d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49058d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56532/0 (socket says 172.21.15.112:56532) 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1079675511 learned_addr learned my addr 172.21.15.112:0/1079675511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5beffd640 1 --2- 172.21.15.112:0/1079675511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f4905a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1079675511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 msgr2=0x55f490588690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1079675511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f490588690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1079675511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 msgr2=0x55f4905a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1079675511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f4905a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.709+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1079675511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49058e780 con 0x55f490592440 2024-04-24T23:14:17.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1079675511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49058d260 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80314d0 tx=0x7fe5a8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1933cc964aaa8282 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5b57fa640 1 -- 172.21.15.112:0/1079675511 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8055070 con 0x55f490592440 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5b57fa640 1 -- 172.21.15.112:0/1079675511 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe5a80037f0 con 0x55f490592440 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5b57fa640 1 -- 172.21.15.112:0/1079675511 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8040050 con 0x55f490592440 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1079675511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 msgr2=0x55f49058d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1079675511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49058d260 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80314d0 tx=0x7fe5a8003f90 comp rx=0 tx=0).stop 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1079675511 shutdown_connections 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1079675511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490597fa0 0x55f490588690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1079675511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f4905a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1079675511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490592440 0x55f49058d260 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1079675511 >> 172.21.15.112:0/1079675511 conn(0x55f49058dab0 msgr2=0x55f49058de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1079675511 shutdown_connections 2024-04-24T23:14:17.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.710+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1079675511 wait complete. 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 1 -- start start 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 1 Processor -- start 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490592440 0x55f49059f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f49059fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490597fa0 0x55f4905a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f4905aa750 con 0x55f490597fa0 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f4905aa080 con 0x55f490592e30 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f4905aae20 con 0x55f490592440 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5c87f8740 10 client.0 _ll_register_callbacks cb 0x7fe5a8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490597fa0 0x55f4905a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5be7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490597fa0 0x55f4905a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56542/0 (socket says 172.21.15.112:56542) 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1764394500 learned_addr learned my addr 172.21.15.112:0/1764394500 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5beffd640 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f49059fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.711+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490592440 0x55f49059f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1764394500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490592440 msgr2=0x55f49059f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490592440 0x55f49059f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1764394500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 msgr2=0x55f49059fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f49059fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5be7fc640 1 -- 172.21.15.112:0/1764394500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4905a3090 con 0x55f490597fa0 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5be7fc640 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490597fa0 0x55f4905a2a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8031570 tx=0x7fe5a8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9991f39a93dae33a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe59effd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe59effd640 1 -- 172.21.15.112:0/1764394500 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8063060 con 0x55f490597fa0 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe59effd640 1 -- 172.21.15.112:0/1764394500 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe5a8004b60 con 0x55f490597fa0 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1764394500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a80037f0 con 0x55f490597fa0 2024-04-24T23:14:17.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe59effd640 1 -- 172.21.15.112:0/1764394500 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe5a8055040 con 0x55f490597fa0 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1764394500 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f49060f410 con 0x55f490597fa0 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe59effd640 1 -- 172.21.15.112:0/1764394500 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe5a80037f0 con 0x55f490597fa0 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe59effd640 1 -- 172.21.15.112:0/1764394500 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe5a80456a0 con 0x55f490597fa0 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 20 client.15291 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f490613b10 0x55f490610620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 4 client.15291 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.712+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1764394500 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "status"}) -- 0x55f4904fd790 con 0x55f490613b10 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5beffd640 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f490613b10 0x55f490610620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe59effd640 10 client.15291 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5beffd640 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f490613b10 0x55f490610620 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c26e13da07fda2f1 server_cookie=f292dfa2c6e8d913 in_seq=0 out_seq=0 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe59effd640 1 -- 172.21.15.112:0/1764394500 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f4904fd790 con 0x55f490613b10 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe59effd640 10 client.15291 handle_command_reply: tid=0 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 client.15291 shutdown 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1764394500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490597fa0 msgr2=0x55f4905a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490597fa0 0x55f4905a2a20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8031570 tx=0x7fe5a8032f90 comp rx=0 tx=0).stop 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1764394500 shutdown_connections 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f490613b10 0x55f490610620 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f490597fa0 0x55f4905a2a20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe59effd640 0 client.15291 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:14:17.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f490592e30 0x55f49059fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 --2- 172.21.15.112:0/1764394500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f490592440 0x55f49059f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1764394500 >> 172.21.15.112:0/1764394500 conn(0x55f49058dab0 msgr2=0x55f490588690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.713+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1764394500 shutdown_connections 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c87f8740 1 -- 172.21.15.112:0/1764394500 wait complete. 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c87f8740 20 client.15291 trim_cache size 0 max 16384 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24490, 2024-04-24T23:14:17.724 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:17.725 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:17.725 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:17.725 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 85.835476846999995, 2024-04-24T23:14:17.725 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:17.725 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 92, 2024-04-24T23:14:17.725 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:17.725 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 124.92866056299999 2024-04-24T23:14:17.725 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5ac0a2b90 msgr2=0x7fe5ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5ac0a2b90 0x7fe5ac0a5050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe5b809ba30 tx=0x7fe5b806eb60 comp rx=0 tx=0).stop 2024-04-24T23:14:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0102230 msgr2=0x7fe5c019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0102230 0x7fe5c019eec0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800c970 tx=0x7fe5b800ce30 comp rx=0 tx=0).stop 2024-04-24T23:14:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3381500298 shutdown_connections 2024-04-24T23:14:17.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe5ac0a2b90 0x7fe5ac0a5050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe5c0102230 0x7fe5c019eec0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5c0101870 0x7fe5c019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 --2- 172.21.15.112:0/3381500298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe5c0100680 0x7fe5c0069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3381500298 >> 172.21.15.112:0/3381500298 conn(0x7fe5c00fbbf0 msgr2=0x7fe5c0104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3381500298 shutdown_connections 2024-04-24T23:14:17.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.714+0000 7fe5c5a12640 1 -- 172.21.15.112:0/3381500298 wait complete. 2024-04-24T23:14:17.728 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:14:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.824+0000 7f534371d640 1 -- 172.21.15.112:0/3978888514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c102230 msgr2=0x7f533c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.824+0000 7f534371d640 1 --2- 172.21.15.112:0/3978888514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c102230 0x7f533c108ac0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5334009900 tx=0x7f533402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.824+0000 7f534371d640 1 -- 172.21.15.112:0/3978888514 shutdown_connections 2024-04-24T23:14:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.824+0000 7f534371d640 1 --2- 172.21.15.112:0/3978888514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c102230 0x7f533c108ac0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.824+0000 7f534371d640 1 --2- 172.21.15.112:0/3978888514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f533c101870 0x7f533c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.824+0000 7f534371d640 1 --2- 172.21.15.112:0/3978888514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f533c100680 0x7f533c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 -- 172.21.15.112:0/3978888514 >> 172.21.15.112:0/3978888514 conn(0x7f533c0fbbf0 msgr2=0x7f533c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 -- 172.21.15.112:0/3978888514 shutdown_connections 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 -- 172.21.15.112:0/3978888514 wait complete. 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 Processor -- start 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 -- start start 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f533c100680 0x7f533c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c101870 0x7f533c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f533c102230 0x7f533c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f533c10c430 con 0x7f533c101870 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f533c10c2b0 con 0x7f533c102230 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f534371d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f533c10c5b0 con 0x7f533c100680 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f5340cb8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c101870 0x7f533c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f5340cb8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c101870 0x7f533c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56564/0 (socket says 172.21.15.112:56564) 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.825+0000 7f5340cb8640 1 -- 172.21.15.112:0/92414283 learned_addr learned my addr 172.21.15.112:0/92414283 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f5341cba640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f533c102230 0x7f533c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f53414b9640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f533c100680 0x7f533c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f5340cb8640 1 -- 172.21.15.112:0/92414283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f533c100680 msgr2=0x7f533c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f5340cb8640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f533c100680 0x7f533c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f5340cb8640 1 -- 172.21.15.112:0/92414283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f533c102230 msgr2=0x7f533c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f5340cb8640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f533c102230 0x7f533c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f5340cb8640 1 -- 172.21.15.112:0/92414283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533c19f720 con 0x7f533c101870 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f5341cba640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f533c102230 0x7f533c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f5340cb8640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c101870 0x7f533c198040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c970 tx=0x7f532c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f53327fc640 1 -- 172.21.15.112:0/92414283 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c007bb0 con 0x7f533c101870 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f53327fc640 1 -- 172.21.15.112:0/92414283 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f532c007d50 con 0x7f533c101870 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f534371d640 1 -- 172.21.15.112:0/92414283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533c19fa10 con 0x7f533c101870 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f53327fc640 1 -- 172.21.15.112:0/92414283 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c00f4b0 con 0x7f533c101870 2024-04-24T23:14:17.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.826+0000 7f534371d640 1 -- 172.21.15.112:0/92414283 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f533c19ff10 con 0x7f533c101870 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.828+0000 7f53327fc640 1 -- 172.21.15.112:0/92414283 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f532c0046c0 con 0x7f533c101870 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5346420740 1 Processor -- start 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5346420740 1 -- start start 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5346420740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 0x562debeac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5346420740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeace30 0x562debeb1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5346420740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeb1fa0 0x562debeb4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5346420740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562debe0ff00 con 0x562debeace30 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53327fc640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53280a2b90 0x7f53280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5346420740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562debdb8750 con 0x562debeb1fa0 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5346420740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562debe0bec0 con 0x562debeac440 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53414b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeace30 0x562debeb1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5341cba640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53280a2b90 0x7f53280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53414b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeace30 0x562debeb1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56580/0 (socket says 172.21.15.112:56580) 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53414b9640 1 -- 172.21.15.112:0/1183637502 learned_addr learned my addr 172.21.15.112:0/1183637502 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53327fc640 1 -- 172.21.15.112:0/92414283 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f532c0cf210 con 0x7f533c101870 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5341cba640 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 0x562debeac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeb1fa0 0x562debeb4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53414b9640 1 -- 172.21.15.112:0/1183637502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 msgr2=0x562debeac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53414b9640 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 0x562debeac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53414b9640 1 -- 172.21.15.112:0/1183637502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeb1fa0 msgr2=0x562debeb4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53414b9640 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeb1fa0 0x562debeb4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f53414b9640 1 -- 172.21.15.112:0/1183637502 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562debb242d0 con 0x562debeace30 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.829+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeb1fa0 0x562debeb4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f5341cba640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53280a2b90 0x7f53280a5050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5334005ab0 tx=0x7f5334032d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f53414b9640 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeace30 0x562debeb1930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5338007eb0 tx=0x7f533800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a7808f4f381303 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f53277fe640 1 -- 172.21.15.112:0/1183637502 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338014070 con 0x562debeace30 2024-04-24T23:14:17.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f53277fe640 1 -- 172.21.15.112:0/1183637502 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5338004d20 con 0x562debeace30 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f53277fe640 1 -- 172.21.15.112:0/1183637502 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338010640 con 0x562debeace30 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f5346420740 1 -- 172.21.15.112:0/1183637502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeace30 msgr2=0x562debeb1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f5346420740 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeace30 0x562debeb1930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5338007eb0 tx=0x7f533800e720 comp rx=0 tx=0).stop 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f5346420740 1 -- 172.21.15.112:0/1183637502 shutdown_connections 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f5346420740 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeb1fa0 0x562debeb4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f5346420740 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeace30 0x562debeb1930 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.830+0000 7f5346420740 1 --2- 172.21.15.112:0/1183637502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 0x562debeac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 -- 172.21.15.112:0/1183637502 >> 172.21.15.112:0/1183637502 conn(0x562debea7ab0 msgr2=0x562debea7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 -- 172.21.15.112:0/1183637502 shutdown_connections 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 -- 172.21.15.112:0/1183637502 wait complete. 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 -- start start 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 Processor -- start 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debeb6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debebdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562debeb4f50 con 0x562debeac440 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562debdcbcf0 con 0x562debeace30 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562debeb50d0 con 0x562debeb1fa0 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5346420740 10 client.0 _ll_register_callbacks cb 0x562debdcb6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f53414b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f53414b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56594/0 (socket says 172.21.15.112:56594) 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f53414b9640 1 -- 172.21.15.112:0/1560693305 learned_addr learned my addr 172.21.15.112:0/1560693305 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.831+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debeb6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f53414b9640 1 -- 172.21.15.112:0/1560693305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 msgr2=0x562debebdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5341cba640 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debebdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f53414b9640 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debebdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f53414b9640 1 -- 172.21.15.112:0/1560693305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 msgr2=0x562debeb6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f53414b9640 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debeb6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f53414b9640 1 -- 172.21.15.112:0/1560693305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562debbee6b0 con 0x562debeac440 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5341cba640 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debebdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f53414b9640 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb6890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5338000f80 tx=0x7f5338007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e6cbcbd70318d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5324ff9640 1 -- 172.21.15.112:0/1560693305 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338014040 con 0x562debeac440 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5324ff9640 1 -- 172.21.15.112:0/1560693305 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5338003970 con 0x562debeac440 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5346420740 1 -- 172.21.15.112:0/1560693305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5338004d20 con 0x562debeac440 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5324ff9640 1 -- 172.21.15.112:0/1560693305 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5338003b70 con 0x562debeac440 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5346420740 10 client.15303 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5346420740 1 -- 172.21.15.112:0/1560693305 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562debec0cf0 con 0x562debeac440 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5324ff9640 1 -- 172.21.15.112:0/1560693305 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5338004d20 con 0x562debeac440 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5324ff9640 1 -- 172.21.15.112:0/1560693305 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f53380184a0 con 0x562debeac440 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.832+0000 7f5346420740 10 client.15303 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.833+0000 7f5346420740 10 client.15303 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.833+0000 7f5346420740 20 client.15303 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.833+0000 7f5346420740 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x562debec31a0 0x562debec4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.833+0000 7f5346420740 4 client.15303 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.833+0000 7f5346420740 1 -- 172.21.15.112:0/1560693305 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562debe1d440 con 0x562debec31a0 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.833+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x562debec31a0 0x562debec4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.833+0000 7f5324ff9640 10 client.15303 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.833+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x562debec31a0 0x562debec4610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa81d3c5b91253b server_cookie=186887ff0bcf2766 in_seq=0 out_seq=0 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5324ff9640 1 -- 172.21.15.112:0/1560693305 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562debe1d440 con 0x562debec31a0 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5324ff9640 10 client.15303 handle_command_reply: tid=0 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 client.15303 shutdown 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 -- 172.21.15.112:0/1560693305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 msgr2=0x562debeb6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb6890 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5338000f80 tx=0x7f5338007bd0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 -- 172.21.15.112:0/1560693305 shutdown_connections 2024-04-24T23:14:17.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x562debec31a0 0x562debec4610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debebdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5324ff9640 0 client.15303 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debeb6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 --2- 172.21.15.112:0/1560693305 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb6890 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 -- 172.21.15.112:0/1560693305 >> 172.21.15.112:0/1560693305 conn(0x562debea7ab0 msgr2=0x562debea6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 -- 172.21.15.112:0/1560693305 shutdown_connections 2024-04-24T23:14:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 1 -- 172.21.15.112:0/1560693305 wait complete. 2024-04-24T23:14:17.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.834+0000 7f5346420740 20 client.15303 trim_cache size 0 max 16384 2024-04-24T23:14:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.852+0000 7f5346420740 1 Processor -- start 2024-04-24T23:14:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.852+0000 7f5346420740 1 -- start start 2024-04-24T23:14:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.852+0000 7f5346420740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 0x562debeab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.852+0000 7f5346420740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debebdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.852+0000 7f5346420740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeb1fa0 0x562debea7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.852+0000 7f5346420740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562debec2950 con 0x562debeb1fa0 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.852+0000 7f5346420740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562debec3020 con 0x562debeace30 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.852+0000 7f5346420740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562debec36f0 con 0x562debeac440 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeb1fa0 0x562debea7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeb1fa0 0x562debea7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56608/0 (socket says 172.21.15.112:56608) 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 -- 172.21.15.112:0/130879730 learned_addr learned my addr 172.21.15.112:0/130879730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5340cb8640 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debebdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f53414b9640 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 0x562debeab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 -- 172.21.15.112:0/130879730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 msgr2=0x562debeab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 0x562debeab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 -- 172.21.15.112:0/130879730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 msgr2=0x562debebdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debebdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 -- 172.21.15.112:0/130879730 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562debea8780 con 0x562debeb1fa0 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5340cb8640 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debebdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:17.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5341cba640 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeb1fa0 0x562debea7e90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5334009900 tx=0x7f533402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a865c037bec8b077 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5326ffd640 1 -- 172.21.15.112:0/130879730 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533404f070 con 0x562debeb1fa0 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5326ffd640 1 -- 172.21.15.112:0/130879730 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53340046c0 con 0x562debeb1fa0 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.853+0000 7f5326ffd640 1 -- 172.21.15.112:0/130879730 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334035040 con 0x562debeb1fa0 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- 172.21.15.112:0/130879730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeb1fa0 msgr2=0x562debea7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeb1fa0 0x562debea7e90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5334009900 tx=0x7f533402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- 172.21.15.112:0/130879730 shutdown_connections 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeb1fa0 0x562debea7e90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debebdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 --2- 172.21.15.112:0/130879730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeac440 0x562debeab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- 172.21.15.112:0/130879730 >> 172.21.15.112:0/130879730 conn(0x562debea7ab0 msgr2=0x562debead7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- 172.21.15.112:0/130879730 shutdown_connections 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- 172.21.15.112:0/130879730 wait complete. 2024-04-24T23:14:17.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- start start 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 Processor -- start 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debf29150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debf2b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562debec4490 con 0x562debeac440 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562debec3dc0 con 0x562debeace30 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562debec4b60 con 0x562debeb1fa0 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.854+0000 7f5346420740 10 client.0 _ll_register_callbacks cb 0x7f533800ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56618/0 (socket says 172.21.15.112:56618) 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 -- 172.21.15.112:0/1573203074 learned_addr learned my addr 172.21.15.112:0/1573203074 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debf29150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5341cba640 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debf2b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 -- 172.21.15.112:0/1573203074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 msgr2=0x562debf2b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debf2b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 -- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 msgr2=0x562debf29150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debf29150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 -- 172.21.15.112:0/1573203074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562debf2bfc0 con 0x562debeac440 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debf29150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f53414b9640 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb0c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5338006010 tx=0x7f53380186c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3572cba531654877 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5324ff9640 1 -- 172.21.15.112:0/1573203074 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338014070 con 0x562debeac440 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5324ff9640 1 -- 172.21.15.112:0/1573203074 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5338003970 con 0x562debeac440 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5346420740 1 -- 172.21.15.112:0/1573203074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53340046c0 con 0x562debeac440 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5324ff9640 1 -- 172.21.15.112:0/1573203074 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f533800ec20 con 0x562debeac440 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5346420740 10 client.15309 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5346420740 1 -- 172.21.15.112:0/1573203074 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562debf2d2d0 con 0x562debeac440 2024-04-24T23:14:17.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.855+0000 7f5324ff9640 1 -- 172.21.15.112:0/1573203074 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f53340046c0 con 0x562debeac440 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5324ff9640 1 -- 172.21.15.112:0/1573203074 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5338010cc0 con 0x562debeac440 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5346420740 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5346420740 10 client.15309 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5346420740 20 client.15309 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5346420740 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x562debf31d30 0x562debf30a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5346420740 4 client.15309 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5346420740 1 -- 172.21.15.112:0/1573203074 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "status"}) -- 0x562debe17790 con 0x562debf31d30 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x562debf31d30 0x562debf30a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5340cb8640 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x562debf31d30 0x562debf30a40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76369a679ccd59eb server_cookie=4610ddf96277e371 in_seq=0 out_seq=0 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.856+0000 7f5324ff9640 10 client.15309 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5324ff9640 1 -- 172.21.15.112:0/1573203074 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x562debe17790 con 0x562debf31d30 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5324ff9640 10 client.15309 handle_command_reply: tid=0 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 client.15309 shutdown 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 -- 172.21.15.112:0/1573203074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 msgr2=0x562debeb0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb0c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5338006010 tx=0x7f53380186c0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 -- 172.21.15.112:0/1573203074 shutdown_connections 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x562debf31d30 0x562debf30a40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562debeb1fa0 0x562debf2b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562debeace30 0x562debf29150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 --2- 172.21.15.112:0/1573203074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562debeac440 0x562debeb0c80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5324ff9640 0 client.15309 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 -- 172.21.15.112:0/1573203074 >> 172.21.15.112:0/1573203074 conn(0x562debea7ab0 msgr2=0x562debea2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 -- 172.21.15.112:0/1573203074 shutdown_connections 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 1 -- 172.21.15.112:0/1573203074 wait complete. 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f5346420740 20 client.15309 trim_cache size 0 max 16384 2024-04-24T23:14:17.866 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24524, 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 109.873312614, 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 92, 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 116.41988434 2024-04-24T23:14:17.867 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 -- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53280a2b90 msgr2=0x7f53280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53280a2b90 0x7f53280a5050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5334005ab0 tx=0x7f5334032d00 comp rx=0 tx=0).stop 2024-04-24T23:14:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 -- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c101870 msgr2=0x7f533c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c101870 0x7f533c198040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c970 tx=0x7f532c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:14:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 -- 172.21.15.112:0/92414283 shutdown_connections 2024-04-24T23:14:17.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f53280a2b90 0x7f53280a5050 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f533c102230 0x7f533c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f533c101870 0x7f533c198040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 --2- 172.21.15.112:0/92414283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f533c100680 0x7f533c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:17.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.857+0000 7f534371d640 1 -- 172.21.15.112:0/92414283 >> 172.21.15.112:0/92414283 conn(0x7f533c0fbbf0 msgr2=0x7f533c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:17.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.858+0000 7f534371d640 1 -- 172.21.15.112:0/92414283 shutdown_connections 2024-04-24T23:14:17.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:17.858+0000 7f534371d640 1 -- 172.21.15.112:0/92414283 wait complete. 2024-04-24T23:14:17.870 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T23:14:17.871 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:14:17.880 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-24T23:14:17.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T23:14:18.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:17 smithi112 ceph-mon[29186]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:18.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:17 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3650763324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:18.128 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:17 smithi187 ceph-mon[33292]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:17 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3650763324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:17 smithi150 ceph-mon[32247]: pgmap v347: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:17 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3650763324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:19.940 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:19 smithi112 ceph-mon[29186]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:20.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 -- 172.21.15.112:0/763467026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 msgr2=0x7f01181025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 --2- 172.21.15.112:0/763467026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 0x7f01181025e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f01140099c0 tx=0x7f0114009cd0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 -- 172.21.15.112:0/763467026 shutdown_connections 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 --2- 172.21.15.112:0/763467026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118103d60 0x7f011810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 --2- 172.21.15.112:0/763467026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 0x7f01181025e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 --2- 172.21.15.112:0/763467026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01181033c0 0x7f0118103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 -- 172.21.15.112:0/763467026 >> 172.21.15.112:0/763467026 conn(0x7f01180fd760 msgr2=0x7f01180ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 -- 172.21.15.112:0/763467026 shutdown_connections 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.042+0000 7f011e59d700 1 -- 172.21.15.112:0/763467026 wait complete. 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011e59d700 1 Processor -- start 2024-04-24T23:14:20.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011e59d700 1 -- start start 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011e59d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 0x7f01181984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011e59d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01181033c0 0x7f01181989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011e59d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118103d60 0x7f011819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011e59d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f011810df00 con 0x7f01181021c0 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011e59d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f011810dd80 con 0x7f0118103d60 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011e59d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f011810e080 con 0x7f01181033c0 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 0x7f01181984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 0x7f01181984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56636/0 (socket says 172.21.15.112:56636) 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 -- 172.21.15.112:0/3028829167 learned_addr learned my addr 172.21.15.112:0/3028829167 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011dd9c700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118103d60 0x7f011819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011cd9a700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01181033c0 0x7f01181989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 -- 172.21.15.112:0/3028829167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01181033c0 msgr2=0x7f01181989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01181033c0 0x7f01181989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 -- 172.21.15.112:0/3028829167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118103d60 msgr2=0x7f011819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118103d60 0x7f011819fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 -- 172.21.15.112:0/3028829167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01181a0100 con 0x7f01181021c0 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.043+0000 7f011d59b700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 0x7f01181984a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0114003860 tx=0x7f0114003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.044+0000 7f010e7fc700 1 -- 172.21.15.112:0/3028829167 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0114027070 con 0x7f01181021c0 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.044+0000 7f010e7fc700 1 -- 172.21.15.112:0/3028829167 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f011400bda0 con 0x7f01181021c0 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.044+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01181a0390 con 0x7f01181021c0 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.044+0000 7f010e7fc700 1 -- 172.21.15.112:0/3028829167 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011400f730 con 0x7f01181021c0 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.044+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01181a0870 con 0x7f01181021c0 2024-04-24T23:14:20.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.045+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0118068f20 con 0x7f01181021c0 2024-04-24T23:14:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.046+0000 7f010e7fc700 1 -- 172.21.15.112:0/3028829167 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f011402a070 con 0x7f01181021c0 2024-04-24T23:14:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.046+0000 7f010e7fc700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f01040a2c20 0x7f01040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.046+0000 7f011cd9a700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f01040a2c20 0x7f01040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.046+0000 7f010e7fc700 1 -- 172.21.15.112:0/3028829167 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f01140d0480 con 0x7f01181021c0 2024-04-24T23:14:20.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.047+0000 7f011cd9a700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f01040a2c20 0x7f01040a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0108005950 tx=0x7f0108005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:20.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.050+0000 7f010e7fc700 1 -- 172.21.15.112:0/3028829167 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0114013070 con 0x7f01181021c0 2024-04-24T23:14:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:19 smithi187 ceph-mon[33292]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.262+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f0118062670 con 0x7f01040a2c20 2024-04-24T23:14:20.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.263+0000 7f010e7fc700 1 -- 172.21.15.112:0/3028829167 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f0118062670 con 0x7f01040a2c20 2024-04-24T23:14:20.265 INFO:teuthology.orchestra.run.smithi112.stdout:Backend: cephadm 2024-04-24T23:14:20.265 INFO:teuthology.orchestra.run.smithi112.stdout:Available: Yes 2024-04-24T23:14:20.265 INFO:teuthology.orchestra.run.smithi112.stdout:Paused: No 2024-04-24T23:14:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f01040a2c20 msgr2=0x7f01040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f01040a2c20 0x7f01040a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0108005950 tx=0x7f0108005eb0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 msgr2=0x7f01181984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 0x7f01181984a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0114003860 tx=0x7f0114003d10 comp rx=0 tx=0).stop 2024-04-24T23:14:20.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 shutdown_connections 2024-04-24T23:14:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f01040a2c20 0x7f01040a50e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0118103d60 0x7f011819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01181021c0 0x7f01181984a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 --2- 172.21.15.112:0/3028829167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01181033c0 0x7f01181989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 >> 172.21.15.112:0/3028829167 conn(0x7f01180fd760 msgr2=0x7f01181066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 shutdown_connections 2024-04-24T23:14:20.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:20.265+0000 7f011e59d700 1 -- 172.21.15.112:0/3028829167 wait complete. 2024-04-24T23:14:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:19 smithi150 ceph-mon[32247]: pgmap v348: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:20.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T23:14:21.140 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:21 smithi187 ceph-mon[33292]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:21 smithi187 ceph-mon[33292]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:22.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:21 smithi112 ceph-mon[29186]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:22.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:21 smithi112 ceph-mon[29186]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:21 smithi150 ceph-mon[32247]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:21 smithi150 ceph-mon[32247]: pgmap v349: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:22.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 -- 172.21.15.112:0/1689030615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103fa0 msgr2=0x7f597810a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:22.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 --2- 172.21.15.112:0/1689030615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103fa0 0x7f597810a850 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f597000b0d0 tx=0x7f597000b3e0 comp rx=0 tx=0).stop 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 -- 172.21.15.112:0/1689030615 shutdown_connections 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 --2- 172.21.15.112:0/1689030615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103fa0 0x7f597810a850 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 --2- 172.21.15.112:0/1689030615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5978100580 0x7f59781009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 --2- 172.21.15.112:0/1689030615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5978103600 0x7f5978103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 -- 172.21.15.112:0/1689030615 >> 172.21.15.112:0/1689030615 conn(0x7f59780fbea0 msgr2=0x7f59780fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 -- 172.21.15.112:0/1689030615 shutdown_connections 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 -- 172.21.15.112:0/1689030615 wait complete. 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.945+0000 7f597df04700 1 Processor -- start 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f597df04700 1 -- start start 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f597df04700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5978100580 0x7f59781986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f597df04700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103600 0x7f5978198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f597df04700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5978103fa0 0x7f597819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:22.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f597df04700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f597810e0c0 con 0x7f5978103600 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f597df04700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f597810df40 con 0x7f5978100580 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f597df04700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f597810e240 con 0x7f5978103fa0 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103600 0x7f5978198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103600 0x7f5978198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49546/0 (socket says 172.21.15.112:49546) 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 -- 172.21.15.112:0/3769076190 learned_addr learned my addr 172.21.15.112:0/3769076190 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5977fff700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5978103fa0 0x7f597819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 -- 172.21.15.112:0/3769076190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5978103fa0 msgr2=0x7f597819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f59777fe700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5978100580 0x7f59781986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5978103fa0 0x7f597819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 -- 172.21.15.112:0/3769076190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5978100580 msgr2=0x7f59781986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5978100580 0x7f59781986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 -- 172.21.15.112:0/3769076190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59781a0320 con 0x7f5978103600 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f59777fe700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5978100580 0x7f59781986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:22.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5976ffd700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103600 0x7f5978198c00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f596c00b560 tx=0x7f596c00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5974ff9700 1 -- 172.21.15.112:0/3769076190 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c015020 con 0x7f5978103600 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59781a0610 con 0x7f5978103600 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5974ff9700 1 -- 172.21.15.112:0/3769076190 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f596c011d30 con 0x7f5978103600 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.946+0000 7f5974ff9700 1 -- 172.21.15.112:0/3769076190 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c00f2e0 con 0x7f5978103600 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.947+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59781a0b50 con 0x7f5978103600 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.948+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5978068f20 con 0x7f5978103600 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.948+0000 7f5974ff9700 1 -- 172.21.15.112:0/3769076190 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f596c018020 con 0x7f5978103600 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.949+0000 7f5974ff9700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f59600a2c20 0x7f59600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.949+0000 7f59777fe700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f59600a2c20 0x7f59600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.949+0000 7f5974ff9700 1 -- 172.21.15.112:0/3769076190 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f596c0cebe0 con 0x7f5978103600 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.949+0000 7f59777fe700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f59600a2c20 0x7f59600a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f597806a720 tx=0x7f5968008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:22.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:22.953+0000 7f5974ff9700 1 -- 172.21.15.112:0/3769076190 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f596c01f020 con 0x7f5978103600 2024-04-24T23:14:23.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.160+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5978062670 con 0x7f59600a2c20 2024-04-24T23:14:23.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.174+0000 7f5974ff9700 1 -- 172.21.15.112:0/3769076190 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f5978062670 con 0x7f59600a2c20 2024-04-24T23:14:23.176 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:14:23.176 INFO:teuthology.orchestra.run.smithi112.stdout:mds.a smithi112 running (2m) 20s ago 2m 17.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 6785c8969dea 2024-04-24T23:14:23.176 INFO:teuthology.orchestra.run.smithi112.stdout:mds.b smithi150 running (2m) 55s ago 2m 26.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 79bd8ab3f0b5 2024-04-24T23:14:23.176 INFO:teuthology.orchestra.run.smithi112.stdout:mds.c smithi187 running (2m) 55s ago 2m 17.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 761366f08672 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.d smithi112 running (2m) 20s ago 2m 16.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 70c3967d1086 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.e smithi150 running (2m) 55s ago 2m 18.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef a40ffab21ec4 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.f smithi187 running (2m) 55s ago 2m 18.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef ea1a12dc3cb2 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.g smithi112 running (2m) 20s ago 2m 18.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 40f6a20ca81b 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.h smithi150 running (2m) 55s ago 2m 16.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef dfeb7077e80a 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.i smithi187 running (2m) 55s ago 2m 17.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef ea9686f71a19 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.j smithi112 running (2m) 20s ago 2m 14.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef b1feabf18044 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.k smithi150 running (2m) 55s ago 2m 19.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0e0e2b7963d 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mds.l smithi187 running (2m) 55s ago 2m 19.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 4e6c70c50489 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.x smithi112 *:9283,8765 running (9m) 20s ago 9m 515M - 19.0.0-2445-gd2c45d97 f68e37b807ef ba5e22e86d48 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.y smithi150 *:8443,8765 running (7m) 55s ago 7m 452M - 19.0.0-2445-gd2c45d97 f68e37b807ef 17588345cc0c 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.z smithi187 *:8443,8765 running (7m) 55s ago 7m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1f38b294df7c 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mon.a smithi112 running (9m) 20s ago 9m 53.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef d2f995aeef1b 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mon.b smithi150 running (8m) 55s ago 8m 38.8M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 80ff5efbf2be 2024-04-24T23:14:23.177 INFO:teuthology.orchestra.run.smithi112.stdout:mon.c smithi187 running (8m) 55s ago 8m 40.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 0dfefceef0c1 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (7m) 20s ago 7m 54.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 20248e5a436a 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi150 running (7m) 55s ago 7m 57.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef d1e55a960d03 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi187 running (6m) 55s ago 6m 55.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 457f4cc6daf6 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi112 running (6m) 20s ago 6m 47.1M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef f5ec225177bf 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi150 running (5m) 55s ago 5m 43.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef e50706f92cf0 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi187 running (5m) 55s ago 5m 55.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 71fadb5b49cb 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 smithi112 running (5m) 20s ago 5m 41.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 33e505153f53 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 smithi150 running (4m) 55s ago 4m 44.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef fe1ff1c1a338 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 smithi187 running (4m) 55s ago 4m 46.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef cfcbb55bc724 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 smithi112 running (3m) 20s ago 3m 51.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef bbb3aec91b7c 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 smithi150 running (3m) 55s ago 3m 52.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef b37882495cf4 2024-04-24T23:14:23.178 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 smithi187 running (2m) 55s ago 2m 41.4M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 0c6ba9e46c74 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f59600a2c20 msgr2=0x7f59600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f59600a2c20 0x7f59600a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f597806a720 tx=0x7f5968008040 comp rx=0 tx=0).stop 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103600 msgr2=0x7f5978198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103600 0x7f5978198c00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f596c00b560 tx=0x7f596c00b920 comp rx=0 tx=0).stop 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 shutdown_connections 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f59600a2c20 0x7f59600a50e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5978100580 0x7f59781986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5978103fa0 0x7f597819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 --2- 172.21.15.112:0/3769076190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5978103600 0x7f5978198c00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.176+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 >> 172.21.15.112:0/3769076190 conn(0x7f59780fbea0 msgr2=0x7f5978106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:23.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.177+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 shutdown_connections 2024-04-24T23:14:23.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:23.177+0000 7f597df04700 1 -- 172.21.15.112:0/3769076190 wait complete. 2024-04-24T23:14:23.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T23:14:24.171 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:23 smithi187 ceph-mon[33292]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:24.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:23 smithi112 ceph-mon[29186]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:23 smithi150 ceph-mon[32247]: pgmap v350: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:25.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:24 smithi187 ceph-mon[33292]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:25.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:24 smithi112 ceph-mon[29186]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:25.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:24 smithi150 ceph-mon[32247]: from='client.15321 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:25.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1574894268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8102730 msgr2=0x7fe0e8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:25.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1574894268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8102730 0x7fe0e8102b50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80099c0 tx=0x7fe0d8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:14:25.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1574894268 shutdown_connections 2024-04-24T23:14:25.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1574894268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0e8101e10 0x7fe0e81096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1574894268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8102730 0x7fe0e8102b50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1574894268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e8101470 0x7fe0e81018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1574894268 >> 172.21.15.112:0/1574894268 conn(0x7fe0e80fbea0 msgr2=0x7fe0e80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1574894268 shutdown_connections 2024-04-24T23:14:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1574894268 wait complete. 2024-04-24T23:14:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 Processor -- start 2024-04-24T23:14:25.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.878+0000 7fe0ee8de700 1 -- start start 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0ee8de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0e8101470 0x7fe0e8100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0ee8de700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8101e10 0x7fe0e819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0ee8de700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e8102730 0x7fe0e81a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0ee8de700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e810cea0 con 0x7fe0e8101e10 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0ee8de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e810cd20 con 0x7fe0e8101470 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0ee8de700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e810d020 con 0x7fe0e8102730 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8101e10 0x7fe0e819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8101e10 0x7fe0e819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49556/0 (socket says 172.21.15.112:49556) 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 -- 172.21.15.112:0/1975425403 learned_addr learned my addr 172.21.15.112:0/1975425403 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e7fff700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0e8101470 0x7fe0e8100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0ece7b700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e8102730 0x7fe0e81a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 -- 172.21.15.112:0/1975425403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e8102730 msgr2=0x7fe0e81a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:25.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e8102730 0x7fe0e81a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 -- 172.21.15.112:0/1975425403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0e8101470 msgr2=0x7fe0e8100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0e8101470 0x7fe0e8100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 -- 172.21.15.112:0/1975425403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e81a2440 con 0x7fe0e8101e10 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.879+0000 7fe0e77fe700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8101e10 0x7fe0e819f710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c9b0 tx=0x7fe0dc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.880+0000 7fe0e57fa700 1 -- 172.21.15.112:0/1975425403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc007b00 con 0x7fe0e8101e10 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.880+0000 7fe0e57fa700 1 -- 172.21.15.112:0/1975425403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe0dc007ca0 con 0x7fe0e8101e10 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.880+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e81a2730 con 0x7fe0e8101e10 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.880+0000 7fe0e57fa700 1 -- 172.21.15.112:0/1975425403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc018780 con 0x7fe0e8101e10 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.880+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81a2cc0 con 0x7fe0e8101e10 2024-04-24T23:14:25.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.881+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e8068f20 con 0x7fe0e8101e10 2024-04-24T23:14:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.882+0000 7fe0e57fa700 1 -- 172.21.15.112:0/1975425403 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe0dc00f450 con 0x7fe0e8101e10 2024-04-24T23:14:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.882+0000 7fe0e57fa700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe0d00a2ca0 0x7fe0d00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.882+0000 7fe0e7fff700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe0d00a2ca0 0x7fe0d00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.882+0000 7fe0e57fa700 1 -- 172.21.15.112:0/1975425403 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe0dc0cf310 con 0x7fe0e8101e10 2024-04-24T23:14:25.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.883+0000 7fe0e7fff700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe0d00a2ca0 0x7fe0d00a5160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8006010 tx=0x7fe0d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:25.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:25.887+0000 7fe0e57fa700 1 -- 172.21.15.112:0/1975425403 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe0dc014030 con 0x7fe0e8101e10 2024-04-24T23:14:26.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.097+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe0e8062670 con 0x7fe0d00a2ca0 2024-04-24T23:14:26.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.102+0000 7fe0e57fa700 1 -- 172.21.15.112:0/1975425403 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe0e8062670 con 0x7fe0d00a2ca0 2024-04-24T23:14:26.104 INFO:teuthology.orchestra.run.smithi112.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T23:14:26.105 INFO:teuthology.orchestra.run.smithi112.stdout:mds.all 12/12 58s ago 2m smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi150=b;smithi150=e;smithi150=h;smithi150=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2024-04-24T23:14:26.105 INFO:teuthology.orchestra.run.smithi112.stdout:mgr 3/3 58s ago 7m smithi112=x;smithi150=y;smithi187=z;count:3 2024-04-24T23:14:26.105 INFO:teuthology.orchestra.run.smithi112.stdout:mon 3/3 58s ago 8m smithi112:172.21.15.112=a;smithi150:172.21.15.150=b;smithi187:172.21.15.187=c;count:3 2024-04-24T23:14:26.105 INFO:teuthology.orchestra.run.smithi112.stdout:osd 12 58s ago - 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe0d00a2ca0 msgr2=0x7fe0d00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe0d00a2ca0 0x7fe0d00a5160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8006010 tx=0x7fe0d8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8101e10 msgr2=0x7fe0e819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8101e10 0x7fe0e819f710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00c9b0 tx=0x7fe0dc00ccc0 comp rx=0 tx=0).stop 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 shutdown_connections 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe0d00a2ca0 0x7fe0d00a5160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe0e8101470 0x7fe0e8100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0e8102730 0x7fe0e81a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 --2- 172.21.15.112:0/1975425403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe0e8101e10 0x7fe0e819f710 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 >> 172.21.15.112:0/1975425403 conn(0x7fe0e80fbea0 msgr2=0x7fe0e81056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 shutdown_connections 2024-04-24T23:14:26.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:26.105+0000 7fe0ee8de700 1 -- 172.21.15.112:0/1975425403 wait complete. 2024-04-24T23:14:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:25 smithi187 ceph-mon[33292]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:25 smithi112 ceph-mon[29186]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:25 smithi150 ceph-mon[32247]: pgmap v351: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:26.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T23:14:27.182 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:27 smithi187 ceph-mon[33292]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:27 smithi187 ceph-mon[33292]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-24T23:14:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-24T23:14:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-04-24T23:14:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-04-24T23:14:28.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:27 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-24T23:14:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:27 smithi112 ceph-mon[29186]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:27 smithi112 ceph-mon[29186]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-24T23:14:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-24T23:14:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-04-24T23:14:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-04-24T23:14:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:27 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-24T23:14:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:27 smithi150 ceph-mon[32247]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:27 smithi150 ceph-mon[32247]: pgmap v352: 129 pgs: 129 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-24T23:14:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-04-24T23:14:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-04-24T23:14:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]: dispatch 2024-04-24T23:14:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:27 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-04-24T23:14:28.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 -- 172.21.15.112:0/1362454388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103ec0 msgr2=0x7f7ba810a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:28.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 --2- 172.21.15.112:0/1362454388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103ec0 0x7f7ba810a770 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000b0a0 tx=0x7f7ba000b3b0 comp rx=0 tx=0).stop 2024-04-24T23:14:28.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 -- 172.21.15.112:0/1362454388 shutdown_connections 2024-04-24T23:14:28.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 --2- 172.21.15.112:0/1362454388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103ec0 0x7f7ba810a770 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:28.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 --2- 172.21.15.112:0/1362454388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba8100540 0x7f7ba8100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:28.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 --2- 172.21.15.112:0/1362454388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8103520 0x7f7ba8103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 -- 172.21.15.112:0/1362454388 >> 172.21.15.112:0/1362454388 conn(0x7f7ba80fbec0 msgr2=0x7f7ba80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 -- 172.21.15.112:0/1362454388 shutdown_connections 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.915+0000 7f7bad322700 1 -- 172.21.15.112:0/1362454388 wait complete. 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7bad322700 1 Processor -- start 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7bad322700 1 -- start start 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7bad322700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba8100540 0x7f7ba81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7bad322700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103520 0x7f7ba81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7bad322700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8103ec0 0x7f7ba819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7bad322700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba810dff0 con 0x7f7ba8103520 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7bad322700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba810de70 con 0x7f7ba8103ec0 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7bad322700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba810e170 con 0x7f7ba8100540 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7ba67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103520 0x7f7ba81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7ba67fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103520 0x7f7ba81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49570/0 (socket says 172.21.15.112:49570) 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7ba67fc700 1 -- 172.21.15.112:0/616638945 learned_addr learned my addr 172.21.15.112:0/616638945 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:28.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7ba67fc700 1 -- 172.21.15.112:0/616638945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba8100540 msgr2=0x7f7ba81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7ba67fc700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba8100540 0x7f7ba81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7ba67fc700 1 -- 172.21.15.112:0/616638945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8103ec0 msgr2=0x7f7ba819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7ba67fc700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8103ec0 0x7f7ba819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.916+0000 7f7ba67fc700 1 -- 172.21.15.112:0/616638945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba81a0110 con 0x7f7ba8103520 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.917+0000 7f7ba67fc700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103520 0x7f7ba81989f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b560 tx=0x7f7b9c00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.917+0000 7f7b8ffff700 1 -- 172.21.15.112:0/616638945 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c015020 con 0x7f7ba8103520 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.917+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba81a0400 con 0x7f7ba8103520 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.917+0000 7f7b8ffff700 1 -- 172.21.15.112:0/616638945 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7b9c011d30 con 0x7f7ba8103520 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.917+0000 7f7b8ffff700 1 -- 172.21.15.112:0/616638945 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00f2e0 con 0x7f7ba8103520 2024-04-24T23:14:28.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.917+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba81a09c0 con 0x7f7ba8103520 2024-04-24T23:14:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.918+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba8068f20 con 0x7f7ba8103520 2024-04-24T23:14:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.919+0000 7f7b8ffff700 1 -- 172.21.15.112:0/616638945 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7b9c018020 con 0x7f7ba8103520 2024-04-24T23:14:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.919+0000 7f7b8ffff700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7b900a2c20 0x7f7b900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.919+0000 7f7ba6ffd700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7b900a2c20 0x7f7b900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.920+0000 7f7b8ffff700 1 -- 172.21.15.112:0/616638945 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9237+0+0 (secure 0 0 0) 0x7f7b9c0cec70 con 0x7f7ba8103520 2024-04-24T23:14:28.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.920+0000 7f7ba6ffd700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7b900a2c20 0x7f7b900a50e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8069d10 tx=0x7f7b98005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:28.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:28.924+0000 7f7b8ffff700 1 -- 172.21.15.112:0/616638945 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7b9c01f020 con 0x7f7ba8103520 2024-04-24T23:14:29.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.132+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7ba8062670 con 0x7f7b900a2c20 2024-04-24T23:14:29.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.134+0000 7f7b8ffff700 1 -- 172.21.15.112:0/616638945 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7ba8062670 con 0x7f7b900a2c20 2024-04-24T23:14:29.135 INFO:teuthology.orchestra.run.smithi112.stdout:HOST ADDR LABELS STATUS 2024-04-24T23:14:29.136 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 172.21.15.112 2024-04-24T23:14:29.136 INFO:teuthology.orchestra.run.smithi112.stdout:smithi150 172.21.15.150 2024-04-24T23:14:29.136 INFO:teuthology.orchestra.run.smithi112.stdout:smithi187 172.21.15.187 2024-04-24T23:14:29.136 INFO:teuthology.orchestra.run.smithi112.stdout:3 hosts in cluster 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7b900a2c20 msgr2=0x7f7b900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7b900a2c20 0x7f7b900a50e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8069d10 tx=0x7f7b98005d20 comp rx=0 tx=0).stop 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103520 msgr2=0x7f7ba81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103520 0x7f7ba81989f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b560 tx=0x7f7b9c00b920 comp rx=0 tx=0).stop 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 shutdown_connections 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7b900a2c20 0x7f7b900a50e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ba8103ec0 0x7f7ba819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba8100540 0x7f7ba81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 --2- 172.21.15.112:0/616638945 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8103520 0x7f7ba81989f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 >> 172.21.15.112:0/616638945 conn(0x7f7ba80fbec0 msgr2=0x7f7ba8106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 shutdown_connections 2024-04-24T23:14:29.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:29.136+0000 7f7bad322700 1 -- 172.21.15.112:0/616638945 wait complete. 2024-04-24T23:14:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-24T23:14:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-24T23:14:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-04-24T23:14:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-04-24T23:14:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:28 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-24T23:14:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:28 smithi187 ceph-mon[33292]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:14:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-24T23:14:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-24T23:14:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-04-24T23:14:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-04-24T23:14:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:28 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-24T23:14:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:28 smithi112 ceph-mon[29186]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:14:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-24T23:14:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-04-24T23:14:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-04-24T23:14:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 10]}]': finished 2024-04-24T23:14:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:28 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-04-24T23:14:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:28 smithi150 ceph-mon[32247]: osdmap e93: 12 total, 12 up, 12 in 2024-04-24T23:14:29.889 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T23:14:30.129 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:29 smithi187 ceph-mon[33292]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T23:14:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:29 smithi187 ceph-mon[33292]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-24T23:14:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:29 smithi187 ceph-mon[33292]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:14:30.251 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:29 smithi112 ceph-mon[29186]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T23:14:30.251 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:29 smithi112 ceph-mon[29186]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-24T23:14:30.251 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:29 smithi112 ceph-mon[29186]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:14:30.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:29 smithi150 ceph-mon[32247]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T23:14:30.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:29 smithi150 ceph-mon[32247]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-24T23:14:30.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:29 smithi150 ceph-mon[32247]: osdmap e94: 12 total, 12 up, 12 in 2024-04-24T23:14:31.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:30 smithi187 ceph-mon[33292]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:31.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:30 smithi112 ceph-mon[29186]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:31.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:30 smithi150 ceph-mon[32247]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3215072336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102390 msgr2=0x7fb3f41027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3215072336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102390 0x7fb3f41027f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80099c0 tx=0x7fb3e8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:14:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3215072336 shutdown_connections 2024-04-24T23:14:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3215072336 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f4102d30 0x7fb3f41095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3215072336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3f4101190 0x7fb3f41015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3215072336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102390 0x7fb3f41027f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3215072336 >> 172.21.15.112:0/3215072336 conn(0x7fb3f40fc730 msgr2=0x7fb3f40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3215072336 shutdown_connections 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3215072336 wait complete. 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.851+0000 7fb3f9aa4700 1 Processor -- start 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f9aa4700 1 -- start start 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f9aa4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f4101190 0x7fb3f41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f9aa4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3f4102390 0x7fb3f4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f9aa4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102d30 0x7fb3f419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f9aa4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f410ceb0 con 0x7fb3f4102d30 2024-04-24T23:14:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f9aa4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f410cd30 con 0x7fb3f4101190 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f9aa4700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f410d030 con 0x7fb3f4102390 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102d30 0x7fb3f419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f3fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102d30 0x7fb3f419fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39022/0 (socket says 172.21.15.112:39022) 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f3fff700 1 -- 172.21.15.112:0/3306053083 learned_addr learned my addr 172.21.15.112:0/3306053083 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f2ffd700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3f4102390 0x7fb3f4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f37fe700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f4101190 0x7fb3f41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f3fff700 1 -- 172.21.15.112:0/3306053083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3f4102390 msgr2=0x7fb3f4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f3fff700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3f4102390 0x7fb3f4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f3fff700 1 -- 172.21.15.112:0/3306053083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f4101190 msgr2=0x7fb3f41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f3fff700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f4101190 0x7fb3f41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.852+0000 7fb3f3fff700 1 -- 172.21.15.112:0/3306053083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f41a01e0 con 0x7fb3f4102d30 2024-04-24T23:14:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.853+0000 7fb3f3fff700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102d30 0x7fb3f419fb40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec007f10 tx=0x7fb3ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.853+0000 7fb3f0ff9700 1 -- 172.21.15.112:0/3306053083 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec004bb0 con 0x7fb3f4102d30 2024-04-24T23:14:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.853+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f41a04d0 con 0x7fb3f4102d30 2024-04-24T23:14:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.853+0000 7fb3f0ff9700 1 -- 172.21.15.112:0/3306053083 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb3ec004d50 con 0x7fb3f4102d30 2024-04-24T23:14:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.853+0000 7fb3f0ff9700 1 -- 172.21.15.112:0/3306053083 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0133f0 con 0x7fb3f4102d30 2024-04-24T23:14:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.853+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f41a0a10 con 0x7fb3f4102d30 2024-04-24T23:14:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.854+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f4068f20 con 0x7fb3f4102d30 2024-04-24T23:14:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.855+0000 7fb3f0ff9700 1 -- 172.21.15.112:0/3306053083 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb3ec01a020 con 0x7fb3f4102d30 2024-04-24T23:14:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.855+0000 7fb3f0ff9700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb3dc0a2c20 0x7fb3dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.855+0000 7fb3f37fe700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb3dc0a2c20 0x7fb3dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.855+0000 7fb3f0ff9700 1 -- 172.21.15.112:0/3306053083 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb3ec0cebb0 con 0x7fb3f4102d30 2024-04-24T23:14:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.856+0000 7fb3f37fe700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb3dc0a2c20 0x7fb3dc0a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb3f41021f0 tx=0x7fb3e400b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:31.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:31.860+0000 7fb3f0ff9700 1 -- 172.21.15.112:0/3306053083 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb3ec010070 con 0x7fb3f4102d30 2024-04-24T23:14:32.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.068+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb3f40fe8b0 con 0x7fb3dc0a2c20 2024-04-24T23:14:32.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.070+0000 7fb3f0ff9700 1 -- 172.21.15.112:0/3306053083 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb3f40fe8b0 con 0x7fb3dc0a2c20 2024-04-24T23:14:32.072 INFO:teuthology.orchestra.run.smithi112.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T23:14:32.072 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T23:14:32.072 INFO:teuthology.orchestra.run.smithi112.stdout:smithi150 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GX400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-24T23:14:32.072 INFO:teuthology.orchestra.run.smithi112.stdout:smithi187 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MW400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-24T23:14:32.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb3dc0a2c20 msgr2=0x7fb3dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:32.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb3dc0a2c20 0x7fb3dc0a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb3f41021f0 tx=0x7fb3e400b5b0 comp rx=0 tx=0).stop 2024-04-24T23:14:32.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102d30 msgr2=0x7fb3f419fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:32.074 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102d30 0x7fb3f419fb40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec007f10 tx=0x7fb3ec00d3b0 comp rx=0 tx=0).stop 2024-04-24T23:14:32.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 shutdown_connections 2024-04-24T23:14:32.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb3dc0a2c20 0x7fb3dc0a50e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:32.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb3f4101190 0x7fb3f41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:32.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb3f4102d30 0x7fb3f419fb40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:32.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 --2- 172.21.15.112:0/3306053083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3f4102390 0x7fb3f4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:32.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 >> 172.21.15.112:0/3306053083 conn(0x7fb3f40fc730 msgr2=0x7fb3f40fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:32.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 shutdown_connections 2024-04-24T23:14:32.075 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:32.072+0000 7fb3f9aa4700 1 -- 172.21.15.112:0/3306053083 wait complete. 2024-04-24T23:14:32.107 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:31 smithi112 ceph-mon[29186]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T23:14:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:31 smithi187 ceph-mon[33292]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T23:14:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:31 smithi150 ceph-mon[32247]: pgmap v356: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-24T23:14:32.725 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:14:32.735 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-24T23:14:32.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T23:14:32.984 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:33 smithi187 ceph-mon[33292]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:33 smithi187 ceph-mon[33292]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-04-24T23:14:34.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:33 smithi112 ceph-mon[29186]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:34.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:33 smithi112 ceph-mon[29186]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-04-24T23:14:34.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:33 smithi150 ceph-mon[32247]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:34.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:33 smithi150 ceph-mon[32247]: pgmap v357: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-04-24T23:14:34.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.743+0000 7f576c5f1700 1 -- 172.21.15.112:0/3523918069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 msgr2=0x7f5764102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:34.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.743+0000 7f576c5f1700 1 --2- 172.21.15.112:0/3523918069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 0x7f5764102810 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f57540099c0 tx=0x7f5754009cd0 comp rx=0 tx=0).stop 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 -- 172.21.15.112:0/3523918069 shutdown_connections 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 --2- 172.21.15.112:0/3523918069 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57641011b0 0x7f57641015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 --2- 172.21.15.112:0/3523918069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5764102d50 0x7f5764109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 --2- 172.21.15.112:0/3523918069 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 0x7f5764102810 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 -- 172.21.15.112:0/3523918069 >> 172.21.15.112:0/3523918069 conn(0x7f57640fc790 msgr2=0x7f57640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 -- 172.21.15.112:0/3523918069 shutdown_connections 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 -- 172.21.15.112:0/3523918069 wait complete. 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 Processor -- start 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 -- start start 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57641011b0 0x7f57641973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 0x7f5764197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5764102d50 0x7f576419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:34.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f576410ced0 con 0x7f57641023b0 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f576410cd50 con 0x7f57641011b0 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f576c5f1700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f576410d050 con 0x7f5764102d50 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.744+0000 7f5769b8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 0x7f5764197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f576a38d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57641011b0 0x7f57641973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f5769b8c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 0x7f5764197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39036/0 (socket says 172.21.15.112:39036) 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f5769b8c700 1 -- 172.21.15.112:0/2967849029 learned_addr learned my addr 172.21.15.112:0/2967849029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f576ab8e700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5764102d50 0x7f576419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f5769b8c700 1 -- 172.21.15.112:0/2967849029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5764102d50 msgr2=0x7f576419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f5769b8c700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5764102d50 0x7f576419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f5769b8c700 1 -- 172.21.15.112:0/2967849029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57641011b0 msgr2=0x7f57641973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f5769b8c700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57641011b0 0x7f57641973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f5769b8c700 1 -- 172.21.15.112:0/2967849029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576419f030 con 0x7f57641023b0 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f576ab8e700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5764102d50 0x7f576419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:34.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f5769b8c700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 0x7f5764197910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f575400b580 tx=0x7f5754005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f575b7fe700 1 -- 172.21.15.112:0/2967849029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754027070 con 0x7f57641023b0 2024-04-24T23:14:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f575b7fe700 1 -- 172.21.15.112:0/2967849029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5754022470 con 0x7f57641023b0 2024-04-24T23:14:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576419f2c0 con 0x7f57641023b0 2024-04-24T23:14:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f575b7fe700 1 -- 172.21.15.112:0/2967849029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575400f6c0 con 0x7f57641023b0 2024-04-24T23:14:34.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.745+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576419f7a0 con 0x7f57641023b0 2024-04-24T23:14:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.746+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5764068f20 con 0x7f57641023b0 2024-04-24T23:14:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.747+0000 7f575b7fe700 1 -- 172.21.15.112:0/2967849029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f575402a070 con 0x7f57641023b0 2024-04-24T23:14:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.748+0000 7f575b7fe700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f57500a2c20 0x7f57500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.748+0000 7f576a38d700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f57500a2c20 0x7f57500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.748+0000 7f575b7fe700 1 -- 172.21.15.112:0/2967849029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f57540d07a0 con 0x7f57641023b0 2024-04-24T23:14:34.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.748+0000 7f576a38d700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f57500a2c20 0x7f57500a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5764102210 tx=0x7f576000a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:34.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.752+0000 7f575b7fe700 1 -- 172.21.15.112:0/2967849029 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5754013070 con 0x7f57641023b0 2024-04-24T23:14:34.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.993+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f576402db60 con 0x7f57641023b0 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.993+0000 7f575b7fe700 1 -- 172.21.15.112:0/2967849029 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3102 (secure 0 0 0) 0x7f575409d3a0 con 0x7f57641023b0 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:e27 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 27 2024-04-24T23:14:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-04-24T23:12:27.330443+0000 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-04-24T23:12:55.876908+0000 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 300 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 0 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 5 2024-04-24T23:14:34.997 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1,2,3,4 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=24524,1=24490,2=24482,3=24425,4=24424} 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3,4] 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data disabled 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 0 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 24524 members: 24524,24482,24490,24425,24424 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b{0:24524} state up:active seq 3 addr [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.998 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c{0:24466} state up:standby-replay seq 1 addr [v2:172.21.15.187:6836/35756805,v1:172.21.15.187:6837/35756805] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.i{1:24490} state up:active seq 11 addr [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f{1:24454} state up:standby-replay seq 1 addr [v2:172.21.15.187:6834/1908796622,v1:172.21.15.187:6835/1908796622] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.h{2:24482} state up:active seq 12 addr [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e{2:24452} state up:standby-replay seq 1 addr [v2:172.21.15.150:6834/2076270934,v1:172.21.15.150:6835/2076270934] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.k{3:24425} state up:active seq 17 addr [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a{3:14640} state up:standby-replay seq 1 addr [v2:172.21.15.112:6840/3500837168,v1:172.21.15.112:6841/3500837168] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.l{4:24424} state up:active seq 18 addr [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.g{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.112:6838/2592086011,v1:172.21.15.112:6839/2592086011] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.j{-1:14556} state up:standby seq 1 addr [v2:172.21.15.112:6834/4200698623,v1:172.21.15.112:6835/4200698623] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:34.999 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d{-1:14568} state up:standby seq 1 addr [v2:172.21.15.112:6836/3896751620,v1:172.21.15.112:6837/3896751620] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:14:35.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f57500a2c20 msgr2=0x7f57500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:35.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f57500a2c20 0x7f57500a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5764102210 tx=0x7f576000a3b0 comp rx=0 tx=0).stop 2024-04-24T23:14:35.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 msgr2=0x7f5764197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:35.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 0x7f5764197910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f575400b580 tx=0x7f5754005ee0 comp rx=0 tx=0).stop 2024-04-24T23:14:35.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 shutdown_connections 2024-04-24T23:14:35.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f57500a2c20 0x7f57500a50e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:35.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57641011b0 0x7f57641973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:35.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5764102d50 0x7f576419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:35.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 --2- 172.21.15.112:0/2967849029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57641023b0 0x7f5764197910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:35.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 >> 172.21.15.112:0/2967849029 conn(0x7f57640fc790 msgr2=0x7f57641055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:35.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 shutdown_connections 2024-04-24T23:14:35.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:34.997+0000 7f576c5f1700 1 -- 172.21.15.112:0/2967849029 wait complete. 2024-04-24T23:14:35.001 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-04-24T23:14:35.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:34 smithi112 ceph-mon[29186]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-24T23:14:35.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:34 smithi112 ceph-mon[29186]: Cluster is now healthy 2024-04-24T23:14:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:34 smithi150 ceph-mon[32247]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-24T23:14:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:34 smithi150 ceph-mon[32247]: Cluster is now healthy 2024-04-24T23:14:35.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:34 smithi187 ceph-mon[33292]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-24T23:14:35.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:34 smithi187 ceph-mon[33292]: Cluster is now healthy 2024-04-24T23:14:35.743 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-24T23:14:35.982 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:36.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:35 smithi112 ceph-mon[29186]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-24T23:14:36.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:35 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2967849029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:14:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:35 smithi150 ceph-mon[32247]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-24T23:14:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:35 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2967849029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:14:36.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:35 smithi187 ceph-mon[33292]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-24T23:14:36.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:35 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2967849029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:14:37.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 -- 172.21.15.112:0/4120296055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0101190 msgr2=0x7fb2f01015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:37.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4120296055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0101190 0x7fb2f01015b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0099c0 tx=0x7fb2ec009cd0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 -- 172.21.15.112:0/4120296055 shutdown_connections 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4120296055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f0102d30 0x7fb2f01095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4120296055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0101190 0x7fb2f01015b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4120296055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2f0102390 0x7fb2f01027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 -- 172.21.15.112:0/4120296055 >> 172.21.15.112:0/4120296055 conn(0x7fb2f00fc730 msgr2=0x7fb2f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 -- 172.21.15.112:0/4120296055 shutdown_connections 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.661+0000 7fb2f8621700 1 -- 172.21.15.112:0/4120296055 wait complete. 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f8621700 1 Processor -- start 2024-04-24T23:14:37.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f8621700 1 -- start start 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f8621700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f0101190 0x7fb2f0105120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f8621700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2f0102390 0x7fb2f0103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f8621700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0102d30 0x7fb2f0103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f8621700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010cf90 con 0x7fb2f0102d30 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f8621700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010ce10 con 0x7fb2f0101190 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f8621700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f010d110 con 0x7fb2f0102390 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f6bbe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0102d30 0x7fb2f0103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f6bbe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0102d30 0x7fb2f0103cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39060/0 (socket says 172.21.15.112:39060) 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f63bd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f0101190 0x7fb2f0105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.662+0000 7fb2f6bbe700 1 -- 172.21.15.112:0/4006218374 learned_addr learned my addr 172.21.15.112:0/4006218374 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f5bbc700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2f0102390 0x7fb2f0103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f6bbe700 1 -- 172.21.15.112:0/4006218374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2f0102390 msgr2=0x7fb2f0103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f6bbe700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2f0102390 0x7fb2f0103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f6bbe700 1 -- 172.21.15.112:0/4006218374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f0101190 msgr2=0x7fb2f0105120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f6bbe700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f0101190 0x7fb2f0105120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f6bbe700 1 -- 172.21.15.112:0/4006218374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f0104560 con 0x7fb2f0102d30 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f5bbc700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2f0102390 0x7fb2f0103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f6bbe700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0102d30 0x7fb2f0103cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800b970 tx=0x7fb2e800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2e77fe700 1 -- 172.21.15.112:0/4006218374 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8019070 con 0x7fb2f0102d30 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2e77fe700 1 -- 172.21.15.112:0/4006218374 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb2e800ce30 con 0x7fb2f0102d30 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f01a2820 con 0x7fb2f0102d30 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2e77fe700 1 -- 172.21.15.112:0/4006218374 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80075d0 con 0x7fb2f0102d30 2024-04-24T23:14:37.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.663+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f01a2d30 con 0x7fb2f0102d30 2024-04-24T23:14:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.664+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f01a2ff0 con 0x7fb2f0102d30 2024-04-24T23:14:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.665+0000 7fb2e77fe700 1 -- 172.21.15.112:0/4006218374 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb2e801c070 con 0x7fb2f0102d30 2024-04-24T23:14:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.666+0000 7fb2e77fe700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2dc0a2c20 0x7fb2dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.666+0000 7fb2f63bd700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2dc0a2c20 0x7fb2dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.666+0000 7fb2e77fe700 1 -- 172.21.15.112:0/4006218374 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb2e80cf250 con 0x7fb2f0102d30 2024-04-24T23:14:37.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.666+0000 7fb2f63bd700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2dc0a2c20 0x7fb2dc0a50e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec006010 tx=0x7fb2ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:37.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.669+0000 7fb2e77fe700 1 -- 172.21.15.112:0/4006218374 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb2e8023070 con 0x7fb2f0102d30 2024-04-24T23:14:37.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.870+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fb2f01a31c0 con 0x7fb2f0102d30 2024-04-24T23:14:37.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.871+0000 7fb2e77fe700 1 -- 172.21.15.112:0/4006218374 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5760 (secure 0 0 0) 0x7fb2e809bdc0 con 0x7fb2f0102d30 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 94 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:fsid e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-04-24T23:04:28.856850+0000 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-04-24T23:14:28.925442+0000 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:crush_version 27 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:full_ratio 0.95 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:backfillfull_ratio 0.9 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:nearfull_ratio 0.85 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:require_min_compat_client luminous 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:min_compat_client luminous 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:require_osd_release squid 2024-04-24T23:14:37.873 INFO:teuthology.orchestra.run.smithi112.stdout:stretch_mode_enabled false 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:max_osd 12 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 up in weight 1 up_from 8 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6802/3410548369,v1:172.21.15.112:6803/3410548369] [v2:172.21.15.112:6804/3410548369,v1:172.21.15.112:6805/3410548369] exists,up 6c7d3b0e-ce0f-4b59-955f-98303b9bd863 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6800/459871910,v1:172.21.15.150:6801/459871910] [v2:172.21.15.150:6802/459871910,v1:172.21.15.150:6803/459871910] exists,up a46ec67e-696c-497b-a0f4-9a38bae79a8f 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6800/3027054636,v1:172.21.15.187:6801/3027054636] [v2:172.21.15.187:6802/3027054636,v1:172.21.15.187:6803/3027054636] exists,up 4899db4a-7c50-4d57-bce9-50d0e840ca6f 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6810/1421446388,v1:172.21.15.112:6811/1421446388] [v2:172.21.15.112:6812/1421446388,v1:172.21.15.112:6813/1421446388] exists,up 6c500d4f-a92d-489a-98c8-758763754114 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 up in weight 1 up_from 33 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6808/3997288444,v1:172.21.15.150:6809/3997288444] [v2:172.21.15.150:6810/3997288444,v1:172.21.15.150:6811/3997288444] exists,up 57a393a7-46ea-440d-b990-315d5806e675 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 up in weight 1 up_from 39 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6808/1092655909,v1:172.21.15.187:6809/1092655909] [v2:172.21.15.187:6810/1092655909,v1:172.21.15.187:6811/1092655909] exists,up 905c0bbc-34be-4035-9799-e3385ea78043 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 up in weight 1 up_from 47 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6818/1354124649,v1:172.21.15.112:6819/1354124649] [v2:172.21.15.112:6820/1354124649,v1:172.21.15.112:6821/1354124649] exists,up 14b8b9f0-b530-4914-a7c6-d3b8b9fa6002 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 up in weight 1 up_from 52 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6816/591791108,v1:172.21.15.150:6817/591791108] [v2:172.21.15.150:6818/591791108,v1:172.21.15.150:6819/591791108] exists,up aacba960-f7c4-41cd-8ee0-cac0fba62c5d 2024-04-24T23:14:37.874 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 up in weight 1 up_from 60 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6816/3574382245,v1:172.21.15.187:6817/3574382245] [v2:172.21.15.187:6818/3574382245,v1:172.21.15.187:6819/3574382245] exists,up c907152a-2f78-4da1-8c5a-853706d3c59f 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 up in weight 1 up_from 68 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6826/3646247852,v1:172.21.15.112:6827/3646247852] [v2:172.21.15.112:6828/3646247852,v1:172.21.15.112:6829/3646247852] exists,up e6e8af2d-787c-472a-973b-da754768b4e6 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 up in weight 1 up_from 73 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.150:6824/599401021,v1:172.21.15.150:6825/599401021] [v2:172.21.15.150:6826/599401021,v1:172.21.15.150:6827/599401021] exists,up 0114d00e-ffc8-4e0d-b79d-c87a604b5cc3 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6824/3702998666,v1:172.21.15.187:6825/3702998666] [v2:172.21.15.187:6826/3702998666,v1:172.21.15.187:6827/3702998666] exists,up 971bddea-134b-4d7e-9ae6-770a9129fdb1 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.0 [8,1] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.b [2,4] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.10 [8,11] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.13 [2,11] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.6 [4,7] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.10 [5,9] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.17 [4,8] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.1b [4,6] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.1d [4,8] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.33 [5,9] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.38 [3,9] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 4.f [9,8] 2024-04-24T23:14:37.875 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 4.12 [5,10] 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 4.17 [9,10] 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 4.19 [9,8] 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1532442772 expires 2024-04-25T23:05:26.702079+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/2220790491 expires 2024-04-25T23:05:26.702079+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/342211680 expires 2024-04-25T23:05:26.702079+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/2319978079 expires 2024-04-25T23:05:26.702079+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/2957535572 expires 2024-04-25T23:05:00.376626+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1556121872 expires 2024-04-25T23:05:00.376626+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/3348191830 expires 2024-04-25T23:05:00.376626+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/3348191830 expires 2024-04-25T23:05:00.376626+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/4091217333 expires 2024-04-25T23:05:00.376626+0000 2024-04-24T23:14:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/2319978079 expires 2024-04-25T23:05:26.702079+0000 2024-04-24T23:14:37.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2dc0a2c20 msgr2=0x7fb2dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:37.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2dc0a2c20 0x7fb2dc0a50e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec006010 tx=0x7fb2ec000bc0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0102d30 msgr2=0x7fb2f0103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:37.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0102d30 0x7fb2f0103cb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800b970 tx=0x7fb2e800bc80 comp rx=0 tx=0).stop 2024-04-24T23:14:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 shutdown_connections 2024-04-24T23:14:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2dc0a2c20 0x7fb2dc0a50e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2f0101190 0x7fb2f0105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f0102d30 0x7fb2f0103cb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 --2- 172.21.15.112:0/4006218374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2f0102390 0x7fb2f0103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 >> 172.21.15.112:0/4006218374 conn(0x7fb2f00fc730 msgr2=0x7fb2f00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 shutdown_connections 2024-04-24T23:14:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:37.874+0000 7fb2f8621700 1 -- 172.21.15.112:0/4006218374 wait complete. 2024-04-24T23:14:38.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:38 smithi112 ceph-mon[29186]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-24T23:14:38.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:38 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4006218374' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:14:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:38 smithi150 ceph-mon[32247]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-24T23:14:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:38 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4006218374' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:14:38.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:38 smithi187 ceph-mon[33292]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-24T23:14:38.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:38 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4006218374' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-24T23:14:38.586 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-24T23:14:38.598 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:38.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 -- 172.21.15.112:0/720655156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34101790 msgr2=0x7f9c34101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:38.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 --2- 172.21.15.112:0/720655156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34101790 0x7f9c34101bb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009900 tx=0x7f9c2c02a410 comp rx=0 tx=0).stop 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 -- 172.21.15.112:0/720655156 shutdown_connections 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 --2- 172.21.15.112:0/720655156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34103340 0x7f9c34109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 --2- 172.21.15.112:0/720655156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c34102980 0x7f9c34102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 --2- 172.21.15.112:0/720655156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34101790 0x7f9c34101bb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 -- 172.21.15.112:0/720655156 >> 172.21.15.112:0/720655156 conn(0x7f9c340fcd20 msgr2=0x7f9c340ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 -- 172.21.15.112:0/720655156 shutdown_connections 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.697+0000 7f9c3984c640 1 -- 172.21.15.112:0/720655156 wait complete. 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c3984c640 1 Processor -- start 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c3984c640 1 -- start start 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c3984c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34101790 0x7f9c34197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c3984c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34102980 0x7f9c34198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c3984c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c34103340 0x7f9c3419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:38.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c3984c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3410d5a0 con 0x7f9c34102980 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c3984c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3410d420 con 0x7f9c34101790 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c3984c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3410d720 con 0x7f9c34103340 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34102980 0x7f9c34198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34102980 0x7f9c34198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39064/0 (socket says 172.21.15.112:39064) 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 -- 172.21.15.112:0/1705210915 learned_addr learned my addr 172.21.15.112:0/1705210915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c337fe640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c34103340 0x7f9c3419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c32ffd640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34101790 0x7f9c34197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 -- 172.21.15.112:0/1705210915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c34103340 msgr2=0x7f9c3419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c34103340 0x7f9c3419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 -- 172.21.15.112:0/1705210915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34101790 msgr2=0x7f9c34197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34101790 0x7f9c34197bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 -- 172.21.15.112:0/1705210915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3419f810 con 0x7f9c34102980 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.698+0000 7f9c327fc640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34102980 0x7f9c34198130 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800ba50 tx=0x7f9c2800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.699+0000 7f9c3884a640 1 -- 172.21.15.112:0/1705210915 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28019070 con 0x7f9c34102980 2024-04-24T23:14:38.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.699+0000 7f9c3884a640 1 -- 172.21.15.112:0/1705210915 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9c2800cbe0 con 0x7f9c34102980 2024-04-24T23:14:38.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.699+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c3419fb00 con 0x7f9c34102980 2024-04-24T23:14:38.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.699+0000 7f9c3884a640 1 -- 172.21.15.112:0/1705210915 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28007770 con 0x7f9c34102980 2024-04-24T23:14:38.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.699+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c341a0000 con 0x7f9c34102980 2024-04-24T23:14:38.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.700+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c34108ab0 con 0x7f9c34102980 2024-04-24T23:14:38.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.701+0000 7f9c3884a640 1 -- 172.21.15.112:0/1705210915 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9c2801c070 con 0x7f9c34102980 2024-04-24T23:14:38.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.701+0000 7f9c3884a640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c1c0a2b90 0x7f9c1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:38.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.701+0000 7f9c32ffd640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c1c0a2b90 0x7f9c1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:38.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.701+0000 7f9c3884a640 1 -- 172.21.15.112:0/1705210915 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f9c280ceab0 con 0x7f9c34102980 2024-04-24T23:14:38.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.702+0000 7f9c32ffd640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c1c0a2b90 0x7f9c1c0a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0316a0 tx=0x7f9c2c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:38.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.706+0000 7f9c3884a640 1 -- 172.21.15.112:0/1705210915 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9c2809afd0 con 0x7f9c34102980 2024-04-24T23:14:38.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.882+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c34062370 con 0x7f9c34102980 2024-04-24T23:14:38.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.884+0000 7f9c3884a640 1 -- 172.21.15.112:0/1705210915 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7f9c2809f2d0 con 0x7f9c34102980 2024-04-24T23:14:38.886 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:38.887 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:38.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c1c0a2b90 msgr2=0x7f9c1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:38.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c1c0a2b90 0x7f9c1c0a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c0316a0 tx=0x7f9c2c000bc0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34102980 msgr2=0x7f9c34198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:38.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34102980 0x7f9c34198130 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800ba50 tx=0x7f9c2800bf10 comp rx=0 tx=0).stop 2024-04-24T23:14:38.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 shutdown_connections 2024-04-24T23:14:38.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9c1c0a2b90 0x7f9c1c0a5050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c34103340 0x7f9c3419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c34102980 0x7f9c34198130 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 --2- 172.21.15.112:0/1705210915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34101790 0x7f9c34197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:38.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 >> 172.21.15.112:0/1705210915 conn(0x7f9c340fcd20 msgr2=0x7f9c34105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:38.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 shutdown_connections 2024-04-24T23:14:38.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.886+0000 7f9c3984c640 1 -- 172.21.15.112:0/1705210915 wait complete. 2024-04-24T23:14:38.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:14:38.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:14:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 -- 172.21.15.112:0/2819054518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c1033e0 msgr2=0x7faa8c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 --2- 172.21.15.112:0/2819054518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c1033e0 0x7faa8c109c70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faa84009cd0 tx=0x7faa8402a8e0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 -- 172.21.15.112:0/2819054518 shutdown_connections 2024-04-24T23:14:39.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 --2- 172.21.15.112:0/2819054518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c1033e0 0x7faa8c109c70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 --2- 172.21.15.112:0/2819054518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa8c102a20 0x7faa8c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 --2- 172.21.15.112:0/2819054518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa8c0ffa50 0x7faa8c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 -- 172.21.15.112:0/2819054518 >> 172.21.15.112:0/2819054518 conn(0x7faa8c0fb380 msgr2=0x7faa8c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 -- 172.21.15.112:0/2819054518 shutdown_connections 2024-04-24T23:14:39.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:38.999+0000 7faa90e15640 1 -- 172.21.15.112:0/2819054518 wait complete. 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa90e15640 1 Processor -- start 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa90e15640 1 -- start start 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa90e15640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa8c0ffa50 0x7faa8c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa90e15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c102a20 0x7faa8c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa90e15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa8c1033e0 0x7faa8c19f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa90e15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c10d5e0 con 0x7faa8c102a20 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa90e15640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c10d460 con 0x7faa8c0ffa50 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa90e15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c10d760 con 0x7faa8c1033e0 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa89d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c102a20 0x7faa8c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa89d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c102a20 0x7faa8c198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39094/0 (socket says 172.21.15.112:39094) 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa89d9b640 1 -- 172.21.15.112:0/152154709 learned_addr learned my addr 172.21.15.112:0/152154709 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa8a59c640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa8c0ffa50 0x7faa8c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.000+0000 7faa8ad9d640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa8c1033e0 0x7faa8c19f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.001+0000 7faa89d9b640 1 -- 172.21.15.112:0/152154709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa8c1033e0 msgr2=0x7faa8c19f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.001+0000 7faa89d9b640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa8c1033e0 0x7faa8c19f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.001+0000 7faa89d9b640 1 -- 172.21.15.112:0/152154709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa8c0ffa50 msgr2=0x7faa8c197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.001+0000 7faa89d9b640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa8c0ffa50 0x7faa8c197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.001+0000 7faa89d9b640 1 -- 172.21.15.112:0/152154709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c19f7f0 con 0x7faa8c102a20 2024-04-24T23:14:39.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.001+0000 7faa89d9b640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c102a20 0x7faa8c198110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c890 tx=0x7faa8000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.002+0000 7faa7b7fe640 1 -- 172.21.15.112:0/152154709 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80004220 con 0x7faa8c102a20 2024-04-24T23:14:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.002+0000 7faa7b7fe640 1 -- 172.21.15.112:0/152154709 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7faa800043c0 con 0x7faa8c102a20 2024-04-24T23:14:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.002+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c19fae0 con 0x7faa8c102a20 2024-04-24T23:14:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.002+0000 7faa7b7fe640 1 -- 172.21.15.112:0/152154709 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80004a00 con 0x7faa8c102a20 2024-04-24T23:14:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.003+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c1a0030 con 0x7faa8c102a20 2024-04-24T23:14:39.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.004+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8c108af0 con 0x7faa8c102a20 2024-04-24T23:14:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.005+0000 7faa7b7fe640 1 -- 172.21.15.112:0/152154709 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7faa80004c60 con 0x7faa8c102a20 2024-04-24T23:14:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.005+0000 7faa7b7fe640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faa740a2b90 0x7faa740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.005+0000 7faa7b7fe640 1 -- 172.21.15.112:0/152154709 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7faa800cf3c0 con 0x7faa8c102a20 2024-04-24T23:14:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.005+0000 7faa8a59c640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faa740a2b90 0x7faa740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.006+0000 7faa8a59c640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faa740a2b90 0x7faa740a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faa8c069b20 tx=0x7faa7c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:39.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.011+0000 7faa7b7fe640 1 -- 172.21.15.112:0/152154709 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faa80014030 con 0x7faa8c102a20 2024-04-24T23:14:39.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.186+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa8c062340 con 0x7faa8c102a20 2024-04-24T23:14:39.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.186+0000 7faa7b7fe640 1 -- 172.21.15.112:0/152154709 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7faa8009bf30 con 0x7faa8c102a20 2024-04-24T23:14:39.188 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:14:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faa740a2b90 msgr2=0x7faa740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faa740a2b90 0x7faa740a5050 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faa8c069b20 tx=0x7faa7c009110 comp rx=0 tx=0).stop 2024-04-24T23:14:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c102a20 msgr2=0x7faa8c198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c102a20 0x7faa8c198110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c890 tx=0x7faa8000cc90 comp rx=0 tx=0).stop 2024-04-24T23:14:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 shutdown_connections 2024-04-24T23:14:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7faa740a2b90 0x7faa740a5050 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faa8c1033e0 0x7faa8c19f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa8c102a20 0x7faa8c198110 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 --2- 172.21.15.112:0/152154709 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa8c0ffa50 0x7faa8c197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 >> 172.21.15.112:0/152154709 conn(0x7faa8c0fb380 msgr2=0x7faa8c105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 shutdown_connections 2024-04-24T23:14:39.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.188+0000 7faa90e15640 1 -- 172.21.15.112:0/152154709 wait complete. 2024-04-24T23:14:39.200 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:39 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1705210915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:39.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:14:39.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.299+0000 7f3810de3640 1 -- 172.21.15.112:0/1065269439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 msgr2=0x7f380c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.299+0000 7f3810de3640 1 --2- 172.21.15.112:0/1065269439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 0x7f380c100aa0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3800009900 tx=0x7f380002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:39.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 -- 172.21.15.112:0/1065269439 shutdown_connections 2024-04-24T23:14:39.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 --2- 172.21.15.112:0/1065269439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f380c102230 0x7f380c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 --2- 172.21.15.112:0/1065269439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f380c101870 0x7f380c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 --2- 172.21.15.112:0/1065269439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 0x7f380c100aa0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 -- 172.21.15.112:0/1065269439 >> 172.21.15.112:0/1065269439 conn(0x7f380c0fbbf0 msgr2=0x7f380c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:39.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 -- 172.21.15.112:0/1065269439 shutdown_connections 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 -- 172.21.15.112:0/1065269439 wait complete. 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 Processor -- start 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 -- start start 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 0x7f380c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f380c101870 0x7f380c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f380c102230 0x7f380c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10d540 con 0x7f380c100680 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10d3c0 con 0x7f380c102230 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f3810de3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f380c10d6c0 con 0x7f380c101870 2024-04-24T23:14:39.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.300+0000 7f380a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 0x7f380c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 0x7f380c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39118/0 (socket says 172.21.15.112:39118) 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380a59c640 1 -- 172.21.15.112:0/2890593452 learned_addr learned my addr 172.21.15.112:0/2890593452 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380ad9d640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f380c102230 0x7f380c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f3809d9b640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f380c101870 0x7f380c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380a59c640 1 -- 172.21.15.112:0/2890593452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f380c101870 msgr2=0x7f380c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380a59c640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f380c101870 0x7f380c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380a59c640 1 -- 172.21.15.112:0/2890593452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f380c102230 msgr2=0x7f380c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380a59c640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f380c102230 0x7f380c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380a59c640 1 -- 172.21.15.112:0/2890593452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c19f6f0 con 0x7f380c100680 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380ad9d640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f380c102230 0x7f380c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f3809d9b640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f380c101870 0x7f380c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:39.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f380a59c640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 0x7f380c197ad0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3800009230 tx=0x7f3800004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:39.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f37fb7fe640 1 -- 172.21.15.112:0/2890593452 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380003f030 con 0x7f380c100680 2024-04-24T23:14:39.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f37fb7fe640 1 -- 172.21.15.112:0/2890593452 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3800003d00 con 0x7f380c100680 2024-04-24T23:14:39.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c19f980 con 0x7f380c100680 2024-04-24T23:14:39.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f37fb7fe640 1 -- 172.21.15.112:0/2890593452 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800035070 con 0x7f380c100680 2024-04-24T23:14:39.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.301+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c19fe70 con 0x7f380c100680 2024-04-24T23:14:39.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.302+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c107940 con 0x7f380c100680 2024-04-24T23:14:39.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.303+0000 7f37fb7fe640 1 -- 172.21.15.112:0/2890593452 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3800042030 con 0x7f380c100680 2024-04-24T23:14:39.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.304+0000 7f37fb7fe640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f37f40a2b90 0x7f37f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.304+0000 7f3809d9b640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f37f40a2b90 0x7f37f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.304+0000 7f37fb7fe640 1 -- 172.21.15.112:0/2890593452 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f38000f0050 con 0x7f380c100680 2024-04-24T23:14:39.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.304+0000 7f3809d9b640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f37f40a2b90 0x7f37f40a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f380c198fe0 tx=0x7f37fc005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:39.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.307+0000 7f37fb7fe640 1 -- 172.21.15.112:0/2890593452 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f38000bcbc0 con 0x7f380c100680 2024-04-24T23:14:39.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:39 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1705210915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:39.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:39 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1705210915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:39.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.486+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f380c02d170 con 0x7f380c100680 2024-04-24T23:14:39.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.486+0000 7f37fb7fe640 1 -- 172.21.15.112:0/2890593452 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f38000bc4c0 con 0x7f380c100680 2024-04-24T23:14:39.488 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f37f40a2b90 msgr2=0x7f37f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f37f40a2b90 0x7f37f40a5050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f380c198fe0 tx=0x7f37fc005d90 comp rx=0 tx=0).stop 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 msgr2=0x7f380c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 0x7f380c197ad0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3800009230 tx=0x7f3800004430 comp rx=0 tx=0).stop 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 shutdown_connections 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f37f40a2b90 0x7f37f40a5050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f380c102230 0x7f380c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f380c101870 0x7f380c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 --2- 172.21.15.112:0/2890593452 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f380c100680 0x7f380c197ad0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 >> 172.21.15.112:0/2890593452 conn(0x7f380c0fbbf0 msgr2=0x7f380c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:39.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 shutdown_connections 2024-04-24T23:14:39.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.488+0000 7f3810de3640 1 -- 172.21.15.112:0/2890593452 wait complete. 2024-04-24T23:14:39.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:14:39.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- 172.21.15.112:0/3847444032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36081006a0 msgr2=0x7f3608100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 --2- 172.21.15.112:0/3847444032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36081006a0 0x7f3608100ac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3604009900 tx=0x7f360402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:39.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- 172.21.15.112:0/3847444032 shutdown_connections 2024-04-24T23:14:39.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 --2- 172.21.15.112:0/3847444032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3608102250 0x7f3608108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 --2- 172.21.15.112:0/3847444032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3608101890 0x7f3608101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 --2- 172.21.15.112:0/3847444032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36081006a0 0x7f3608100ac0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- 172.21.15.112:0/3847444032 >> 172.21.15.112:0/3847444032 conn(0x7f36080fbc10 msgr2=0x7f36080fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- 172.21.15.112:0/3847444032 shutdown_connections 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- 172.21.15.112:0/3847444032 wait complete. 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 Processor -- start 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- start start 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36081006a0 0x7f3608196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3608101890 0x7f3608197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3608102250 0x7f360819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f360810c450 con 0x7f3608102250 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f360810c2d0 con 0x7f3608101890 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.601+0000 7f3610570640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f360810c5d0 con 0x7f36081006a0 2024-04-24T23:14:39.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3608102250 0x7f360819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3608102250 0x7f360819e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39126/0 (socket says 172.21.15.112:39126) 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 -- 172.21.15.112:0/2504262246 learned_addr learned my addr 172.21.15.112:0/2504262246 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360e30c640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36081006a0 0x7f3608196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 -- 172.21.15.112:0/2504262246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36081006a0 msgr2=0x7f3608196af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36081006a0 0x7f3608196af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360db0b640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3608101890 0x7f3608197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 -- 172.21.15.112:0/2504262246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3608101890 msgr2=0x7f3608197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3608101890 0x7f3608197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 -- 172.21.15.112:0/2504262246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360819e710 con 0x7f3608102250 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360db0b640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3608101890 0x7f3608197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f360eb0d640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3608102250 0x7f360819e0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3600009fd0 tx=0x7f3600007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f35ff7fe640 1 -- 172.21.15.112:0/2504262246 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360000de10 con 0x7f3608102250 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f35ff7fe640 1 -- 172.21.15.112:0/2504262246 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3600007ca0 con 0x7f3608102250 2024-04-24T23:14:39.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3608069da0 con 0x7f3608102250 2024-04-24T23:14:39.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f35ff7fe640 1 -- 172.21.15.112:0/2504262246 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600018420 con 0x7f3608102250 2024-04-24T23:14:39.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.602+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360806a2a0 con 0x7f3608102250 2024-04-24T23:14:39.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.603+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36080689b0 con 0x7f3608102250 2024-04-24T23:14:39.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.604+0000 7f35ff7fe640 1 -- 172.21.15.112:0/2504262246 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3600004000 con 0x7f3608102250 2024-04-24T23:14:39.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.605+0000 7f35ff7fe640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35f40a2b90 0x7f35f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.605+0000 7f360e30c640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35f40a2b90 0x7f35f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.605+0000 7f35ff7fe640 1 -- 172.21.15.112:0/2504262246 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f36000d10e0 con 0x7f3608102250 2024-04-24T23:14:39.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.605+0000 7f360e30c640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35f40a2b90 0x7f35f40a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3604006010 tx=0x7f3604005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:39.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.608+0000 7f35ff7fe640 1 -- 172.21.15.112:0/2504262246 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f360009dc50 con 0x7f3608102250 2024-04-24T23:14:39.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.785+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3608107670 con 0x7f3608102250 2024-04-24T23:14:39.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.785+0000 7f35ff7fe640 1 -- 172.21.15.112:0/2504262246 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f360009d550 con 0x7f3608102250 2024-04-24T23:14:39.787 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:14:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35f40a2b90 msgr2=0x7f35f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35f40a2b90 0x7f35f40a5050 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3604006010 tx=0x7f3604005be0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3608102250 msgr2=0x7f360819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3608102250 0x7f360819e0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3600009fd0 tx=0x7f3600007590 comp rx=0 tx=0).stop 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 shutdown_connections 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35f40a2b90 0x7f35f40a5050 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3608102250 0x7f360819e0a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3608101890 0x7f3608197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 --2- 172.21.15.112:0/2504262246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36081006a0 0x7f3608196af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 >> 172.21.15.112:0/2504262246 conn(0x7f36080fbc10 msgr2=0x7f3608104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 shutdown_connections 2024-04-24T23:14:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.787+0000 7f3610570640 1 -- 172.21.15.112:0/2504262246 wait complete. 2024-04-24T23:14:39.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:14:39.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.898+0000 7f68aaf63640 1 -- 172.21.15.112:0/3744802269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4100680 msgr2=0x7f68a4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.898+0000 7f68aaf63640 1 --2- 172.21.15.112:0/3744802269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4100680 0x7f68a4100aa0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6898009900 tx=0x7f689802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:39.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.898+0000 7f68aaf63640 1 -- 172.21.15.112:0/3744802269 shutdown_connections 2024-04-24T23:14:39.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.898+0000 7f68aaf63640 1 --2- 172.21.15.112:0/3744802269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a4102230 0x7f68a4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.898+0000 7f68aaf63640 1 --2- 172.21.15.112:0/3744802269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a4101870 0x7f68a4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.898+0000 7f68aaf63640 1 --2- 172.21.15.112:0/3744802269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4100680 0x7f68a4100aa0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 -- 172.21.15.112:0/3744802269 >> 172.21.15.112:0/3744802269 conn(0x7f68a40fbbf0 msgr2=0x7f68a40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 -- 172.21.15.112:0/3744802269 shutdown_connections 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 -- 172.21.15.112:0/3744802269 wait complete. 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 Processor -- start 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 -- start start 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a4100680 0x7f68a4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4101870 0x7f68a41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a4102230 0x7f68a419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68a410d540 con 0x7f68a4101870 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f68a410d3c0 con 0x7f68a4102230 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68aaf63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f68a410d6c0 con 0x7f68a4100680 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4101870 0x7f68a41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4101870 0x7f68a41980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39144/0 (socket says 172.21.15.112:39144) 2024-04-24T23:14:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68a3fff640 1 -- 172.21.15.112:0/1530678988 learned_addr learned my addr 172.21.15.112:0/1530678988 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.899+0000 7f68a9500640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a4102230 0x7f68a419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a3fff640 1 -- 172.21.15.112:0/1530678988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a4100680 msgr2=0x7f68a4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a3fff640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a4100680 0x7f68a4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a3fff640 1 -- 172.21.15.112:0/1530678988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a4102230 msgr2=0x7f68a419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a3fff640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a4102230 0x7f68a419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a3fff640 1 -- 172.21.15.112:0/1530678988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a419f7a0 con 0x7f68a4101870 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a3fff640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4101870 0x7f68a41980c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f689000d770 tx=0x7f689000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a1ffb640 1 -- 172.21.15.112:0/1530678988 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f68a4101870 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a1ffb640 1 -- 172.21.15.112:0/1530678988 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6890005520 con 0x7f68a4101870 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a419fa90 con 0x7f68a4101870 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68a1ffb640 1 -- 172.21.15.112:0/1530678988 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890010460 con 0x7f68a4101870 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.900+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a419ff90 con 0x7f68a4101870 2024-04-24T23:14:39.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.901+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a41079a0 con 0x7f68a4101870 2024-04-24T23:14:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.902+0000 7f68a1ffb640 1 -- 172.21.15.112:0/1530678988 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6890004ea0 con 0x7f68a4101870 2024-04-24T23:14:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.902+0000 7f68a1ffb640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f68940a2b90 0x7f68940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.903+0000 7f68a8cff640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f68940a2b90 0x7f68940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.903+0000 7f68a1ffb640 1 -- 172.21.15.112:0/1530678988 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f68900cf030 con 0x7f68a4101870 2024-04-24T23:14:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.903+0000 7f68a8cff640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f68940a2b90 0x7f68940a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6898005c20 tx=0x7f6898000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:39.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:39.907+0000 7f68a1ffb640 1 -- 172.21.15.112:0/1530678988 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f689009bba0 con 0x7f68a4101870 2024-04-24T23:14:40.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.091+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f68a4062340 con 0x7f68a4101870 2024-04-24T23:14:40.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.091+0000 7f68a1ffb640 1 -- 172.21.15.112:0/1530678988 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f689009b4a0 con 0x7f68a4101870 2024-04-24T23:14:40.093 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f68940a2b90 msgr2=0x7f68940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f68940a2b90 0x7f68940a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6898005c20 tx=0x7f6898000bc0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4101870 msgr2=0x7f68a41980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4101870 0x7f68a41980c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f689000d770 tx=0x7f689000dc30 comp rx=0 tx=0).stop 2024-04-24T23:14:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 shutdown_connections 2024-04-24T23:14:40.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f68940a2b90 0x7f68940a5050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68a4102230 0x7f68a419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a4101870 0x7f68a41980c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 --2- 172.21.15.112:0/1530678988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68a4100680 0x7f68a4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 >> 172.21.15.112:0/1530678988 conn(0x7f68a40fbbf0 msgr2=0x7f68a4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 shutdown_connections 2024-04-24T23:14:40.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.093+0000 7f68aaf63640 1 -- 172.21.15.112:0/1530678988 wait complete. 2024-04-24T23:14:40.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:40 smithi112 ceph-mon[29186]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T23:14:40.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:40 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/152154709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:40 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2890593452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:40 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2504262246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.106 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 -- 172.21.15.112:0/1611086843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6840ffe60 msgr2=0x7fa684100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1611086843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6840ffe60 0x7fa684100280 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa674009900 tx=0x7fa67402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 -- 172.21.15.112:0/1611086843 shutdown_connections 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1611086843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa684101a00 0x7fa684108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1611086843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa684101040 0x7fa6841014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1611086843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa6840ffe60 0x7fa684100280 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 -- 172.21.15.112:0/1611086843 >> 172.21.15.112:0/1611086843 conn(0x7fa6840fb420 msgr2=0x7fa6840fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 -- 172.21.15.112:0/1611086843 shutdown_connections 2024-04-24T23:14:40.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.202+0000 7fa688c3f640 1 -- 172.21.15.112:0/1611086843 wait complete. 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa688c3f640 1 Processor -- start 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa688c3f640 1 -- start start 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa688c3f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6840ffe60 0x7fa684197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa688c3f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa684101040 0x7fa6841981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa688c3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684101a00 0x7fa68419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa688c3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa68410bc80 con 0x7fa684101a00 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa688c3f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa68410bb00 con 0x7fa6840ffe60 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa688c3f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa68410be00 con 0x7fa684101040 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684101a00 0x7fa68419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa682d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684101a00 0x7fa68419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39160/0 (socket says 172.21.15.112:39160) 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa682d9d640 1 -- 172.21.15.112:0/1144910924 learned_addr learned my addr 172.21.15.112:0/1144910924 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa681d9b640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa684101040 0x7fa6841981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa682d9d640 1 -- 172.21.15.112:0/1144910924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa684101040 msgr2=0x7fa6841981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa682d9d640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa684101040 0x7fa6841981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa682d9d640 1 -- 172.21.15.112:0/1144910924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6840ffe60 msgr2=0x7fa684197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa682d9d640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6840ffe60 0x7fa684197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa682d9d640 1 -- 172.21.15.112:0/1144910924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68419f870 con 0x7fa684101a00 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.203+0000 7fa681d9b640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa684101040 0x7fa6841981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.204+0000 7fa682d9d640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684101a00 0x7fa68419f1d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00eaa0 tx=0x7fa67c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.204+0000 7fa6737fe640 1 -- 172.21.15.112:0/1144910924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c00cdb0 con 0x7fa684101a00 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.204+0000 7fa6737fe640 1 -- 172.21.15.112:0/1144910924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa67c004b20 con 0x7fa684101a00 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.204+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68419fb60 con 0x7fa684101a00 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.204+0000 7fa6737fe640 1 -- 172.21.15.112:0/1144910924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c011760 con 0x7fa684101a00 2024-04-24T23:14:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.204+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6841a00b0 con 0x7fa684101a00 2024-04-24T23:14:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.205+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6841a02e0 con 0x7fa684101a00 2024-04-24T23:14:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.206+0000 7fa6737fe640 1 -- 172.21.15.112:0/1144910924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa67c004cd0 con 0x7fa684101a00 2024-04-24T23:14:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.206+0000 7fa6737fe640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa66c0a2b90 0x7fa66c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.206+0000 7fa68259c640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa66c0a2b90 0x7fa66c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.206+0000 7fa6737fe640 1 -- 172.21.15.112:0/1144910924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa67c0cf7d0 con 0x7fa684101a00 2024-04-24T23:14:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.207+0000 7fa68259c640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa66c0a2b90 0x7fa66c0a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa674005c20 tx=0x7fa674000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.211+0000 7fa6737fe640 1 -- 172.21.15.112:0/1144910924 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa67c016030 con 0x7fa684101a00 2024-04-24T23:14:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:40 smithi150 ceph-mon[32247]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T23:14:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:40 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/152154709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:40 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2890593452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:40 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2504262246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.387 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:40 smithi187 ceph-mon[33292]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-24T23:14:40.387 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:40 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/152154709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.388 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:40 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2890593452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.388 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:40 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2504262246' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.421+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6841a0550 con 0x7fa684101a00 2024-04-24T23:14:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.422+0000 7fa6737fe640 1 -- 172.21.15.112:0/1144910924 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11947 (secure 0 0 0) 0x7fa67c09c340 con 0x7fa684101a00 2024-04-24T23:14:40.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:40.424 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:12:55.876908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24482,24490,24425,24424]},"id":1}]} 2024-04-24T23:14:40.425 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-04-24T23:14:40.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa66c0a2b90 msgr2=0x7fa66c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa66c0a2b90 0x7fa66c0a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa674005c20 tx=0x7fa674000bc0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684101a00 msgr2=0x7fa68419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684101a00 0x7fa68419f1d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa67c00eaa0 tx=0x7fa67c00ef60 comp rx=0 tx=0).stop 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 shutdown_connections 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa66c0a2b90 0x7fa66c0a5050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa684101a00 0x7fa68419f1d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa684101040 0x7fa6841981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 --2- 172.21.15.112:0/1144910924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6840ffe60 0x7fa684197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 >> 172.21.15.112:0/1144910924 conn(0x7fa6840fb420 msgr2=0x7fa6840fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 shutdown_connections 2024-04-24T23:14:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.424+0000 7fa688c3f640 1 -- 172.21.15.112:0/1144910924 wait complete. 2024-04-24T23:14:40.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 -- 172.21.15.112:0/2660794918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc100700 msgr2=0x7f2ffc100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 --2- 172.21.15.112:0/2660794918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc100700 0x7f2ffc100b20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009900 tx=0x7f2ff802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 -- 172.21.15.112:0/2660794918 shutdown_connections 2024-04-24T23:14:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 --2- 172.21.15.112:0/2660794918 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ffc1022b0 0x7f2ffc108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 --2- 172.21.15.112:0/2660794918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ffc1018f0 0x7f2ffc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 --2- 172.21.15.112:0/2660794918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc100700 0x7f2ffc100b20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 -- 172.21.15.112:0/2660794918 >> 172.21.15.112:0/2660794918 conn(0x7f2ffc0fbc90 msgr2=0x7f2ffc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:40.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 -- 172.21.15.112:0/2660794918 shutdown_connections 2024-04-24T23:14:40.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 -- 172.21.15.112:0/2660794918 wait complete. 2024-04-24T23:14:40.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.533+0000 7f3003809640 1 Processor -- start 2024-04-24T23:14:40.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 -- start start 2024-04-24T23:14:40.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ffc100700 0x7f2ffc06a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc1018f0 0x7f2ffc19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ffc1022b0 0x7f2ffc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10c440 con 0x7f2ffc1018f0 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10c2c0 con 0x7f2ffc1022b0 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc10c5c0 con 0x7f2ffc100700 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc1018f0 0x7f2ffc19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc1018f0 0x7f2ffc19c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50060/0 (socket says 172.21.15.112:50060) 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 -- 172.21.15.112:0/2544548276 learned_addr learned my addr 172.21.15.112:0/2544548276 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f30015a5640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ffc100700 0x7f2ffc06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 -- 172.21.15.112:0/2544548276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ffc100700 msgr2=0x7f2ffc06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ffc100700 0x7f2ffc06a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 -- 172.21.15.112:0/2544548276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ffc1022b0 msgr2=0x7f2ffc19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3001da6640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ffc1022b0 0x7f2ffc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ffc1022b0 0x7f2ffc19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 -- 172.21.15.112:0/2544548276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc19f7a0 con 0x7f2ffc1018f0 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3000da4640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc1018f0 0x7f2ffc19c870 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00e9f0 tx=0x7f2fec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:40.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f2ff27fc640 1 -- 172.21.15.112:0/2544548276 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec00cd80 con 0x7f2ffc1018f0 2024-04-24T23:14:40.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f2ff27fc640 1 -- 172.21.15.112:0/2544548276 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2fec005520 con 0x7f2ffc1018f0 2024-04-24T23:14:40.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc19fa30 con 0x7f2ffc1018f0 2024-04-24T23:14:40.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f2ff27fc640 1 -- 172.21.15.112:0/2544548276 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec010690 con 0x7f2ffc1018f0 2024-04-24T23:14:40.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.534+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc19ff80 con 0x7f2ffc1018f0 2024-04-24T23:14:40.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.535+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc1079c0 con 0x7f2ffc1018f0 2024-04-24T23:14:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.536+0000 7f2ff27fc640 1 -- 172.21.15.112:0/2544548276 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2fec0046c0 con 0x7f2ffc1018f0 2024-04-24T23:14:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.537+0000 7f2ff27fc640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2fe80a2c10 0x7f2fe80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.537+0000 7f30015a5640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2fe80a2c10 0x7f2fe80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.537+0000 7f2ff27fc640 1 -- 172.21.15.112:0/2544548276 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f2fec0cf860 con 0x7f2ffc1018f0 2024-04-24T23:14:40.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.537+0000 7f30015a5640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2fe80a2c10 0x7f2fe80a50d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8031570 tx=0x7f2ff8031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:40.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.541+0000 7f2ff27fc640 1 -- 172.21.15.112:0/2544548276 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2fec09c3d0 con 0x7f2ffc1018f0 2024-04-24T23:14:40.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.717+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ffc062340 con 0x7f2ffc1018f0 2024-04-24T23:14:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.719+0000 7f2ff27fc640 1 -- 172.21.15.112:0/2544548276 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7f2fec09bcd0 con 0x7f2ffc1018f0 2024-04-24T23:14:40.721 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:40.721 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2fe80a2c10 msgr2=0x7f2fe80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2fe80a2c10 0x7f2fe80a50d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8031570 tx=0x7f2ff8031640 comp rx=0 tx=0).stop 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc1018f0 msgr2=0x7f2ffc19c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc1018f0 0x7f2ffc19c870 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00e9f0 tx=0x7f2fec00edf0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 shutdown_connections 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2fe80a2c10 0x7f2fe80a50d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ffc1022b0 0x7f2ffc19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ffc1018f0 0x7f2ffc19c870 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 --2- 172.21.15.112:0/2544548276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ffc100700 0x7f2ffc06a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 >> 172.21.15.112:0/2544548276 conn(0x7f2ffc0fbc90 msgr2=0x7f2ffc104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 shutdown_connections 2024-04-24T23:14:40.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.721+0000 7f3003809640 1 -- 172.21.15.112:0/2544548276 wait complete. 2024-04-24T23:14:40.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:14:40.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:14:40.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 -- 172.21.15.112:0/1993672988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110102230 msgr2=0x7f4110108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 --2- 172.21.15.112:0/1993672988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110102230 0x7f4110108ac0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4108009900 tx=0x7f410802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 -- 172.21.15.112:0/1993672988 shutdown_connections 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 --2- 172.21.15.112:0/1993672988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110102230 0x7f4110108ac0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 --2- 172.21.15.112:0/1993672988 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4110101870 0x7f4110101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 --2- 172.21.15.112:0/1993672988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4110100680 0x7f4110100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 -- 172.21.15.112:0/1993672988 >> 172.21.15.112:0/1993672988 conn(0x7f41100fbbf0 msgr2=0x7f41100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 -- 172.21.15.112:0/1993672988 shutdown_connections 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.833+0000 7f4114aee640 1 -- 172.21.15.112:0/1993672988 wait complete. 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f4114aee640 1 Processor -- start 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f4114aee640 1 -- start start 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f4114aee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110100680 0x7f4110197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f4114aee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4110101870 0x7f4110198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f4114aee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4110102230 0x7f411019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f4114aee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f411010d520 con 0x7f4110100680 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f4114aee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f411010d3a0 con 0x7f4110101870 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f4114aee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f411010d6a0 con 0x7f4110102230 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f410e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110100680 0x7f4110197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f410e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110100680 0x7f4110197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50072/0 (socket says 172.21.15.112:50072) 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.834+0000 7f410e59c640 1 -- 172.21.15.112:0/2539310609 learned_addr learned my addr 172.21.15.112:0/2539310609 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f410dd9b640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4110101870 0x7f4110198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f410e59c640 1 -- 172.21.15.112:0/2539310609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4110102230 msgr2=0x7f411019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f410e59c640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4110102230 0x7f411019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f410e59c640 1 -- 172.21.15.112:0/2539310609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4110101870 msgr2=0x7f4110198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f410e59c640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4110101870 0x7f4110198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f410e59c640 1 -- 172.21.15.112:0/2539310609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411019f770 con 0x7f4110100680 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f410e59c640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110100680 0x7f4110197b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f410000ccb0 tx=0x7f410000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:40.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f40ff7fe640 1 -- 172.21.15.112:0/2539310609 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4100003b70 con 0x7f4110100680 2024-04-24T23:14:40.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f40ff7fe640 1 -- 172.21.15.112:0/2539310609 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f410000ae50 con 0x7f4110100680 2024-04-24T23:14:40.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411019fa60 con 0x7f4110100680 2024-04-24T23:14:40.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f40ff7fe640 1 -- 172.21.15.112:0/2539310609 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410000f8b0 con 0x7f4110100680 2024-04-24T23:14:40.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.835+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41101a0000 con 0x7f4110100680 2024-04-24T23:14:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.836+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41101078d0 con 0x7f4110100680 2024-04-24T23:14:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.837+0000 7f40ff7fe640 1 -- 172.21.15.112:0/2539310609 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f410000fa50 con 0x7f4110100680 2024-04-24T23:14:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.838+0000 7f40ff7fe640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40f80a2c10 0x7f40f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.838+0000 7f410dd9b640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40f80a2c10 0x7f40f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.838+0000 7f40ff7fe640 1 -- 172.21.15.112:0/2539310609 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f41000cf4d0 con 0x7f4110100680 2024-04-24T23:14:40.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.838+0000 7f410dd9b640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40f80a2c10 0x7f40f80a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4110199060 tx=0x7f410400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:40.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:40.842+0000 7f40ff7fe640 1 -- 172.21.15.112:0/2539310609 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4100014070 con 0x7f4110100680 2024-04-24T23:14:41.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.022+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f411002d170 con 0x7f4110100680 2024-04-24T23:14:41.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.023+0000 7f40ff7fe640 1 -- 172.21.15.112:0/2539310609 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f411002d170 con 0x7f4110100680 2024-04-24T23:14:41.025 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:14:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40f80a2c10 msgr2=0x7f40f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40f80a2c10 0x7f40f80a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4110199060 tx=0x7f410400b390 comp rx=0 tx=0).stop 2024-04-24T23:14:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110100680 msgr2=0x7f4110197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110100680 0x7f4110197b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f410000ccb0 tx=0x7f410000a620 comp rx=0 tx=0).stop 2024-04-24T23:14:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 shutdown_connections 2024-04-24T23:14:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f40f80a2c10 0x7f40f80a50d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4110102230 0x7f411019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4110101870 0x7f4110198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 --2- 172.21.15.112:0/2539310609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4110100680 0x7f4110197b50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 >> 172.21.15.112:0/2539310609 conn(0x7f41100fbbf0 msgr2=0x7f4110104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:41.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 shutdown_connections 2024-04-24T23:14:41.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.025+0000 7f4114aee640 1 -- 172.21.15.112:0/2539310609 wait complete. 2024-04-24T23:14:41.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:14:41.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 -- 172.21.15.112:0/602026206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 msgr2=0x7ff8a4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/602026206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 0x7ff8a4100230 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff898009900 tx=0x7ff89802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:41.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 -- 172.21.15.112:0/602026206 shutdown_connections 2024-04-24T23:14:41.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/602026206 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a4101920 0x7ff8a4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/602026206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a4100f60 0x7ff8a41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/602026206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 0x7ff8a4100230 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 -- 172.21.15.112:0/602026206 >> 172.21.15.112:0/602026206 conn(0x7ff8a40fb380 msgr2=0x7ff8a40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 -- 172.21.15.112:0/602026206 shutdown_connections 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.137+0000 7ff8aaa72640 1 -- 172.21.15.112:0/602026206 wait complete. 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 Processor -- start 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 -- start start 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 0x7ff8a4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a4100f60 0x7ff8a4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a4101920 0x7ff8a419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a410bd40 con 0x7ff8a40ffe10 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a410bbc0 con 0x7ff8a4100f60 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a410bec0 con 0x7ff8a4101920 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 0x7ff8a4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 0x7ff8a4197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50090/0 (socket says 172.21.15.112:50090) 2024-04-24T23:14:41.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 -- 172.21.15.112:0/3910248858 learned_addr learned my addr 172.21.15.112:0/3910248858 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a900f640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a4101920 0x7ff8a419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a3fff640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a4100f60 0x7ff8a4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 -- 172.21.15.112:0/3910248858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a4101920 msgr2=0x7ff8a419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a4101920 0x7ff8a419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 -- 172.21.15.112:0/3910248858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a4100f60 msgr2=0x7ff8a4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a4100f60 0x7ff8a4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 -- 172.21.15.112:0/3910248858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a419f960 con 0x7ff8a40ffe10 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a880e640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 0x7ff8a4197d50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff8980316a0 tx=0x7ff898032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a1ffb640 1 -- 172.21.15.112:0/3910248858 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89803f030 con 0x7ff8a40ffe10 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a1ffb640 1 -- 172.21.15.112:0/3910248858 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff898003d00 con 0x7ff8a40ffe10 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a419fbf0 con 0x7ff8a40ffe10 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.138+0000 7ff8a1ffb640 1 -- 172.21.15.112:0/3910248858 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898035070 con 0x7ff8a40ffe10 2024-04-24T23:14:41.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.139+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a41a0130 con 0x7ff8a40ffe10 2024-04-24T23:14:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.140+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a41a0300 con 0x7ff8a40ffe10 2024-04-24T23:14:41.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.141+0000 7ff8a1ffb640 1 -- 172.21.15.112:0/3910248858 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff898042030 con 0x7ff8a40ffe10 2024-04-24T23:14:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.141+0000 7ff8a1ffb640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8940a2c10 0x7ff8940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.141+0000 7ff8a3fff640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8940a2c10 0x7ff8940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.141+0000 7ff8a1ffb640 1 -- 172.21.15.112:0/3910248858 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff8980efa80 con 0x7ff8a40ffe10 2024-04-24T23:14:41.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.142+0000 7ff8a3fff640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8940a2c10 0x7ff8940a50d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40ffb30 tx=0x7ff890009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:41.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.146+0000 7ff8a1ffb640 1 -- 172.21.15.112:0/3910248858 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff8980bc5f0 con 0x7ff8a40ffe10 2024-04-24T23:14:41.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1530678988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:41.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1144910924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:41.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2544548276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1530678988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1144910924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2544548276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:41.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.322+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff8a41a04d0 con 0x7ff8a40ffe10 2024-04-24T23:14:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.322+0000 7ff8a1ffb640 1 -- 172.21.15.112:0/3910248858 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7ff8980bbef0 con 0x7ff8a40ffe10 2024-04-24T23:14:41.324 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:41.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8940a2c10 msgr2=0x7ff8940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8940a2c10 0x7ff8940a50d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40ffb30 tx=0x7ff890009110 comp rx=0 tx=0).stop 2024-04-24T23:14:41.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 msgr2=0x7ff8a4197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 0x7ff8a4197d50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff8980316a0 tx=0x7ff898032d90 comp rx=0 tx=0).stop 2024-04-24T23:14:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 shutdown_connections 2024-04-24T23:14:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8940a2c10 0x7ff8940a50d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a4101920 0x7ff8a419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a4100f60 0x7ff8a4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 --2- 172.21.15.112:0/3910248858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a40ffe10 0x7ff8a4197d50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 >> 172.21.15.112:0/3910248858 conn(0x7ff8a40fb380 msgr2=0x7ff8a40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 shutdown_connections 2024-04-24T23:14:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.324+0000 7ff8aaa72640 1 -- 172.21.15.112:0/3910248858 wait complete. 2024-04-24T23:14:41.337 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:14:41.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 -- 172.21.15.112:0/3594714227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c4102a20 msgr2=0x7f35c4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 --2- 172.21.15.112:0/3594714227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c4102a20 0x7f35c4102ea0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009930 tx=0x7f35b802a420 comp rx=0 tx=0).stop 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 -- 172.21.15.112:0/3594714227 shutdown_connections 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 --2- 172.21.15.112:0/3594714227 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c41033e0 0x7f35c4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 --2- 172.21.15.112:0/3594714227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c4102a20 0x7f35c4102ea0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 --2- 172.21.15.112:0/3594714227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c40ffa50 0x7f35c40ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 -- 172.21.15.112:0/3594714227 >> 172.21.15.112:0/3594714227 conn(0x7f35c40fb380 msgr2=0x7f35c40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 -- 172.21.15.112:0/3594714227 shutdown_connections 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.436+0000 7f35c9f22640 1 -- 172.21.15.112:0/3594714227 wait complete. 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c9f22640 1 Processor -- start 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c9f22640 1 -- start start 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c9f22640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c40ffa50 0x7f35c4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c9f22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c4102a20 0x7f35c4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c9f22640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c41033e0 0x7f35c419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c9f22640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410d560 con 0x7f35c41033e0 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c9f22640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410d3e0 con 0x7f35c40ffa50 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c9f22640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410d6e0 con 0x7f35c4102a20 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c41033e0 0x7f35c419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c41033e0 0x7f35c419f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50116/0 (socket says 172.21.15.112:50116) 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c3fff640 1 -- 172.21.15.112:0/863648262 learned_addr learned my addr 172.21.15.112:0/863648262 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c2ffd640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c4102a20 0x7f35c4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c37fe640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c40ffa50 0x7f35c4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c3fff640 1 -- 172.21.15.112:0/863648262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c4102a20 msgr2=0x7f35c4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c3fff640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c4102a20 0x7f35c4198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c3fff640 1 -- 172.21.15.112:0/863648262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c40ffa50 msgr2=0x7f35c4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c3fff640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c40ffa50 0x7f35c4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c3fff640 1 -- 172.21.15.112:0/863648262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c419f740 con 0x7f35c41033e0 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.437+0000 7f35c37fe640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c40ffa50 0x7f35c4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:41.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.438+0000 7f35c3fff640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c41033e0 0x7f35c419f0d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009fd0 tx=0x7f35bc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.438+0000 7f35c0ff9640 1 -- 172.21.15.112:0/863648262 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc017070 con 0x7f35c41033e0 2024-04-24T23:14:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.438+0000 7f35c0ff9640 1 -- 172.21.15.112:0/863648262 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f35bc004020 con 0x7f35c41033e0 2024-04-24T23:14:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.438+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c419fa30 con 0x7f35c41033e0 2024-04-24T23:14:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.438+0000 7f35c0ff9640 1 -- 172.21.15.112:0/863648262 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc0106c0 con 0x7f35c41033e0 2024-04-24T23:14:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.438+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c419ff00 con 0x7f35c41033e0 2024-04-24T23:14:41.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.439+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c4108af0 con 0x7f35c41033e0 2024-04-24T23:14:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.440+0000 7f35c0ff9640 1 -- 172.21.15.112:0/863648262 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f35bc0041d0 con 0x7f35c41033e0 2024-04-24T23:14:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.440+0000 7f35c0ff9640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35ac0a2b90 0x7f35ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.440+0000 7f35c37fe640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35ac0a2b90 0x7f35ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.440+0000 7f35c0ff9640 1 -- 172.21.15.112:0/863648262 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f35bc0cf230 con 0x7f35c41033e0 2024-04-24T23:14:41.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.441+0000 7f35c37fe640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35ac0a2b90 0x7f35ac0a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35c4069bc0 tx=0x7f35b4005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:41.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.445+0000 7f35c0ff9640 1 -- 172.21.15.112:0/863648262 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f35bc09bda0 con 0x7f35c41033e0 2024-04-24T23:14:41.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1530678988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:41.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1144910924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:41.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2544548276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:41.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.621+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f35c40623e0 con 0x7f35c41033e0 2024-04-24T23:14:41.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.622+0000 7f35c0ff9640 1 -- 172.21.15.112:0/863648262 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f35bc09b6a0 con 0x7f35c41033e0 2024-04-24T23:14:41.624 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35ac0a2b90 msgr2=0x7f35ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35ac0a2b90 0x7f35ac0a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35c4069bc0 tx=0x7f35b4005db0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c41033e0 msgr2=0x7f35c419f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c41033e0 0x7f35c419f0d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009fd0 tx=0x7f35bc00b260 comp rx=0 tx=0).stop 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 shutdown_connections 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f35ac0a2b90 0x7f35ac0a5050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35c41033e0 0x7f35c419f0d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35c4102a20 0x7f35c4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 --2- 172.21.15.112:0/863648262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35c40ffa50 0x7f35c4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 >> 172.21.15.112:0/863648262 conn(0x7f35c40fb380 msgr2=0x7f35c4105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:41.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 shutdown_connections 2024-04-24T23:14:41.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.624+0000 7f35c9f22640 1 -- 172.21.15.112:0/863648262 wait complete. 2024-04-24T23:14:41.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:14:41.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 -- 172.21.15.112:0/819961411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4104b30 msgr2=0x7efec4106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 --2- 172.21.15.112:0/819961411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4104b30 0x7efec4106f20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efeb8009900 tx=0x7efeb802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 -- 172.21.15.112:0/819961411 shutdown_connections 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 --2- 172.21.15.112:0/819961411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efec4107460 0x7efec41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 --2- 172.21.15.112:0/819961411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4104b30 0x7efec4106f20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 --2- 172.21.15.112:0/819961411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efec4102200 0x7efec41045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 -- 172.21.15.112:0/819961411 >> 172.21.15.112:0/819961411 conn(0x7efec40fbbf0 msgr2=0x7efec40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 -- 172.21.15.112:0/819961411 shutdown_connections 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.735+0000 7efecae31640 1 -- 172.21.15.112:0/819961411 wait complete. 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efecae31640 1 Processor -- start 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efecae31640 1 -- start start 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efecae31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efec4102200 0x7efec4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efecae31640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efec4104b30 0x7efec4198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efecae31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4107460 0x7efec419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efecae31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efec410d530 con 0x7efec4107460 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efecae31640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efec410d3b0 con 0x7efec4104b30 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efecae31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efec410d6b0 con 0x7efec4102200 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec93ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4107460 0x7efec419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec93ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4107460 0x7efec419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50122/0 (socket says 172.21.15.112:50122) 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec93ce640 1 -- 172.21.15.112:0/1315370529 learned_addr learned my addr 172.21.15.112:0/1315370529 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec8bcd640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efec4102200 0x7efec4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec3fff640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efec4104b30 0x7efec4198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec93ce640 1 -- 172.21.15.112:0/1315370529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efec4102200 msgr2=0x7efec4197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec93ce640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efec4102200 0x7efec4197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec93ce640 1 -- 172.21.15.112:0/1315370529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efec4104b30 msgr2=0x7efec4198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec93ce640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efec4104b30 0x7efec4198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec93ce640 1 -- 172.21.15.112:0/1315370529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec419f810 con 0x7efec4107460 2024-04-24T23:14:41.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.736+0000 7efec8bcd640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efec4102200 0x7efec4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:41.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.737+0000 7efec93ce640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4107460 0x7efec419f1a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efebc00cad0 tx=0x7efebc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:41.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.737+0000 7efec1ffb640 1 -- 172.21.15.112:0/1315370529 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc00ae20 con 0x7efec4107460 2024-04-24T23:14:41.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.737+0000 7efec1ffb640 1 -- 172.21.15.112:0/1315370529 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efebc004b20 con 0x7efec4107460 2024-04-24T23:14:41.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.737+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec419fb00 con 0x7efec4107460 2024-04-24T23:14:41.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.737+0000 7efec1ffb640 1 -- 172.21.15.112:0/1315370529 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efebc00f780 con 0x7efec4107460 2024-04-24T23:14:41.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.737+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec419ffd0 con 0x7efec4107460 2024-04-24T23:14:41.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.738+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec40fd6e0 con 0x7efec4107460 2024-04-24T23:14:41.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.739+0000 7efec1ffb640 1 -- 172.21.15.112:0/1315370529 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efebc004cd0 con 0x7efec4107460 2024-04-24T23:14:41.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.739+0000 7efec1ffb640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efeac0a2b90 0x7efeac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:41.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.740+0000 7efec8bcd640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efeac0a2b90 0x7efeac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:41.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.740+0000 7efec1ffb640 1 -- 172.21.15.112:0/1315370529 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7efebc0cf580 con 0x7efec4107460 2024-04-24T23:14:41.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.740+0000 7efec8bcd640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efeac0a2b90 0x7efeac0a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efeb40058c0 tx=0x7efeb4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:41.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.744+0000 7efec1ffb640 1 -- 172.21.15.112:0/1315370529 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efebc014030 con 0x7efec4107460 2024-04-24T23:14:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.921+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7efec4062340 con 0x7efec4107460 2024-04-24T23:14:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.921+0000 7efec1ffb640 1 -- 172.21.15.112:0/1315370529 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7efebc09c0f0 con 0x7efec4107460 2024-04-24T23:14:41.924 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:41.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efeac0a2b90 msgr2=0x7efeac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efeac0a2b90 0x7efeac0a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efeb40058c0 tx=0x7efeb4005850 comp rx=0 tx=0).stop 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4107460 msgr2=0x7efec419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4107460 0x7efec419f1a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efebc00cad0 tx=0x7efebc00cf90 comp rx=0 tx=0).stop 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 shutdown_connections 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efeac0a2b90 0x7efeac0a5050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efec4107460 0x7efec419f1a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efec4104b30 0x7efec4198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 --2- 172.21.15.112:0/1315370529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efec4102200 0x7efec4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.923+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 >> 172.21.15.112:0/1315370529 conn(0x7efec40fbbf0 msgr2=0x7efec4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.924+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 shutdown_connections 2024-04-24T23:14:41.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:41.924+0000 7efecae31640 1 -- 172.21.15.112:0/1315370529 wait complete. 2024-04-24T23:14:41.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 -- 172.21.15.112:0/1474173981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4102230 msgr2=0x7f79f4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 --2- 172.21.15.112:0/1474173981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4102230 0x7f79f4108ac0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009ca0 tx=0x7f79ec02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 -- 172.21.15.112:0/1474173981 shutdown_connections 2024-04-24T23:14:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 --2- 172.21.15.112:0/1474173981 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4102230 0x7f79f4108ac0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 --2- 172.21.15.112:0/1474173981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f4101870 0x7f79f4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 --2- 172.21.15.112:0/1474173981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f4100680 0x7f79f4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 -- 172.21.15.112:0/1474173981 >> 172.21.15.112:0/1474173981 conn(0x7f79f40fbbf0 msgr2=0x7f79f40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 -- 172.21.15.112:0/1474173981 shutdown_connections 2024-04-24T23:14:42.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.033+0000 7f79f9238640 1 -- 172.21.15.112:0/1474173981 wait complete. 2024-04-24T23:14:42.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f9238640 1 Processor -- start 2024-04-24T23:14:42.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f9238640 1 -- start start 2024-04-24T23:14:42.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f9238640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f4100680 0x7f79f4197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f9238640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4101870 0x7f79f4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f9238640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f4102230 0x7f79f419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f9238640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410c420 con 0x7f79f4101870 2024-04-24T23:14:42.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f9238640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410c2a0 con 0x7f79f4102230 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f9238640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79f410c5a0 con 0x7f79f4100680 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4101870 0x7f79f4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4101870 0x7f79f4198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50138/0 (socket says 172.21.15.112:50138) 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.034+0000 7f79f259c640 1 -- 172.21.15.112:0/802791993 learned_addr learned my addr 172.21.15.112:0/802791993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f2d9d640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f4100680 0x7f79f4197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f359e640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f4102230 0x7f79f419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f259c640 1 -- 172.21.15.112:0/802791993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f4100680 msgr2=0x7f79f4197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f259c640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f4100680 0x7f79f4197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f259c640 1 -- 172.21.15.112:0/802791993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f4102230 msgr2=0x7f79f419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f259c640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f4102230 0x7f79f419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f259c640 1 -- 172.21.15.112:0/802791993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f419f710 con 0x7f79f4101870 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f359e640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f4102230 0x7f79f419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f259c640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4101870 0x7f79f4198030 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f79e400e9f0 tx=0x7f79e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79dbfff640 1 -- 172.21.15.112:0/802791993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e400cd80 con 0x7f79f4101870 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79dbfff640 1 -- 172.21.15.112:0/802791993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f79e4010930 con 0x7f79f4101870 2024-04-24T23:14:42.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f419fa00 con 0x7f79f4101870 2024-04-24T23:14:42.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79dbfff640 1 -- 172.21.15.112:0/802791993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4018b00 con 0x7f79f4101870 2024-04-24T23:14:42.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.035+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f419ff50 con 0x7f79f4101870 2024-04-24T23:14:42.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.036+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f41a01e0 con 0x7f79f4101870 2024-04-24T23:14:42.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.037+0000 7f79dbfff640 1 -- 172.21.15.112:0/802791993 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f79e4018ca0 con 0x7f79f4101870 2024-04-24T23:14:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.038+0000 7f79dbfff640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f79dc0a2b90 0x7f79dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.038+0000 7f79f2d9d640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f79dc0a2b90 0x7f79dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.038+0000 7f79dbfff640 1 -- 172.21.15.112:0/802791993 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f79e40cf810 con 0x7f79f4101870 2024-04-24T23:14:42.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.038+0000 7f79f2d9d640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f79dc0a2b90 0x7f79dc0a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005fd0 tx=0x7f79e800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.042+0000 7f79dbfff640 1 -- 172.21.15.112:0/802791993 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f79e409c380 con 0x7f79f4101870 2024-04-24T23:14:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.254+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79f41a03b0 con 0x7f79f4101870 2024-04-24T23:14:42.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.255+0000 7f79dbfff640 1 -- 172.21.15.112:0/802791993 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11947 (secure 0 0 0) 0x7f79e409bc80 con 0x7f79f4101870 2024-04-24T23:14:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:42.257 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:12:55.876908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24482,24490,24425,24424]},"id":1}]} 2024-04-24T23:14:42.257 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 27 2024-04-24T23:14:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f79dc0a2b90 msgr2=0x7f79dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f79dc0a2b90 0x7f79dc0a5050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f79e8005fd0 tx=0x7f79e800b410 comp rx=0 tx=0).stop 2024-04-24T23:14:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4101870 msgr2=0x7f79f4198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4101870 0x7f79f4198030 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f79e400e9f0 tx=0x7f79e400eeb0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 shutdown_connections 2024-04-24T23:14:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f79dc0a2b90 0x7f79dc0a5050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f4102230 0x7f79f419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79f4101870 0x7f79f4198030 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 --2- 172.21.15.112:0/802791993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79f4100680 0x7f79f4197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 >> 172.21.15.112:0/802791993 conn(0x7f79f40fbbf0 msgr2=0x7f79f40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 shutdown_connections 2024-04-24T23:14:42.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.257+0000 7f79f9238640 1 -- 172.21.15.112:0/802791993 wait complete. 2024-04-24T23:14:42.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:42 smithi112 ceph-mon[29186]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:42.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2539310609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3910248858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/863648262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.269 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1315370529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.270 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:42 smithi150 ceph-mon[32247]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2539310609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3910248858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/863648262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1315370529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 -- 172.21.15.112:0/216294933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e8101890 msgr2=0x7f63e8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/216294933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e8101890 0x7f63e8101d10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009900 tx=0x7f63d802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:42.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 -- 172.21.15.112:0/216294933 shutdown_connections 2024-04-24T23:14:42.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/216294933 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e8102250 0x7f63e8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/216294933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e8101890 0x7f63e8101d10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/216294933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63e81006a0 0x7f63e8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 -- 172.21.15.112:0/216294933 >> 172.21.15.112:0/216294933 conn(0x7f63e80fbc30 msgr2=0x7f63e80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 -- 172.21.15.112:0/216294933 shutdown_connections 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 -- 172.21.15.112:0/216294933 wait complete. 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 Processor -- start 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 -- start start 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e81006a0 0x7f63e8196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.367+0000 7f63ef6c4640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e8101890 0x7f63e8197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ef6c4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63e8102250 0x7f63e819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ef6c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63e810c450 con 0x7f63e81006a0 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ef6c4640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f63e810c2d0 con 0x7f63e8101890 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ef6c4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63e810c5d0 con 0x7f63e8102250 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e81006a0 0x7f63e8196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e81006a0 0x7f63e8196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50158/0 (socket says 172.21.15.112:50158) 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 -- 172.21.15.112:0/2110042075 learned_addr learned my addr 172.21.15.112:0/2110042075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63edc61640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63e8102250 0x7f63e819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ecc5f640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e8101890 0x7f63e8197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 -- 172.21.15.112:0/2110042075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63e8102250 msgr2=0x7f63e819e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63e8102250 0x7f63e819e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 -- 172.21.15.112:0/2110042075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e8101890 msgr2=0x7f63e8197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e8101890 0x7f63e8197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 -- 172.21.15.112:0/2110042075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e819e6e0 con 0x7f63e81006a0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ecc5f640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e8101890 0x7f63e8197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ed460640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e81006a0 0x7f63e8196ac0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009850 tx=0x7f63e4009d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63de7fc640 1 -- 172.21.15.112:0/2110042075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e401d070 con 0x7f63e81006a0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e8069e00 con 0x7f63e81006a0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63de7fc640 1 -- 172.21.15.112:0/2110042075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f63e4007ca0 con 0x7f63e81006a0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63de7fc640 1 -- 172.21.15.112:0/2110042075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4016320 con 0x7f63e81006a0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.368+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e806a2d0 con 0x7f63e81006a0 2024-04-24T23:14:42.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.369+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e80689b0 con 0x7f63e81006a0 2024-04-24T23:14:42.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.370+0000 7f63de7fc640 1 -- 172.21.15.112:0/2110042075 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f63e4020070 con 0x7f63e81006a0 2024-04-24T23:14:42.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.371+0000 7f63de7fc640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f63d40a2b90 0x7f63d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.371+0000 7f63ecc5f640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f63d40a2b90 0x7f63d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.371+0000 7f63de7fc640 1 -- 172.21.15.112:0/2110042075 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f63e40d0f30 con 0x7f63e81006a0 2024-04-24T23:14:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.371+0000 7f63ecc5f640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f63d40a2b90 0x7f63d40a5050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d8031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.374+0000 7f63de7fc640 1 -- 172.21.15.112:0/2110042075 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f63e4027070 con 0x7f63e81006a0 2024-04-24T23:14:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:42 smithi187 ceph-mon[33292]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2539310609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3910248858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/863648262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1315370529' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:42.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.551+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63e81076c0 con 0x7f63e81006a0 2024-04-24T23:14:42.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.552+0000 7f63de7fc640 1 -- 172.21.15.112:0/2110042075 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7f63e409daa0 con 0x7f63e81006a0 2024-04-24T23:14:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:42.555 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f63d40a2b90 msgr2=0x7f63d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f63d40a2b90 0x7f63d40a5050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c00 tx=0x7f63d8031640 comp rx=0 tx=0).stop 2024-04-24T23:14:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e81006a0 msgr2=0x7f63e8196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e81006a0 0x7f63e8196ac0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009850 tx=0x7f63e4009d10 comp rx=0 tx=0).stop 2024-04-24T23:14:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 shutdown_connections 2024-04-24T23:14:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f63d40a2b90 0x7f63d40a5050 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63e8102250 0x7f63e819e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f63e8101890 0x7f63e8197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 --2- 172.21.15.112:0/2110042075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e81006a0 0x7f63e8196ac0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 >> 172.21.15.112:0/2110042075 conn(0x7f63e80fbc30 msgr2=0x7f63e8104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.554+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 shutdown_connections 2024-04-24T23:14:42.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.555+0000 7f63ef6c4640 1 -- 172.21.15.112:0/2110042075 wait complete. 2024-04-24T23:14:42.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-24T23:12:27.330443+0000', 'modified': '2024-04-24T23:12:55.876908+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24490, 'mds_2': 24482, 'mds_3': 24425, 'mds_4': 24424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6839/2592086011', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6838', 'nonce': 2592086011}, {'type': 'v1', 'addr': '172.21.15.112:6839', 'nonce': 2592086011}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.112:6841/3500837168', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6840', 'nonce': 3500837168}, {'type': 'v1', 'addr': '172.21.15.112:6841', 'nonce': 3500837168}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24424': {'gid': 24424, 'name': 'l', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.187:6833/1998066775', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1998066775}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 1998066775}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.150:6833/68455606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6832', 'nonce': 68455606}, {'type': 'v1', 'addr': '172.21.15.150:6833', 'nonce': 68455606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.150:6835/2076270934', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6834', 'nonce': 2076270934}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 2076270934}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/1908796622', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 1908796622}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 1908796622}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6837/35756805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 35756805}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 35756805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.150:6837/3592489674', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6836', 'nonce': 3592489674}, {'type': 'v1', 'addr': '172.21.15.150:6837', 'nonce': 3592489674}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.187:6839/3343164580', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 3343164580}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 3343164580}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6839/748900426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6838', 'nonce': 748900426}, {'type': 'v1', 'addr': '172.21.15.150:6839', 'nonce': 748900426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24524, 24482, 24490, 24425, 24424]} 2024-04-24T23:14:42.568 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-24T23:14:42.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/1680185416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0103330 msgr2=0x7f2ea0109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/1680185416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0103330 0x7f2ea0109bc0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009ca0 tx=0x7f2e9802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/1680185416 shutdown_connections 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/1680185416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0103330 0x7f2ea0109bc0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/1680185416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea0102970 0x7f2ea0102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/1680185416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ea0101780 0x7f2ea0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/1680185416 >> 172.21.15.112:0/1680185416 conn(0x7f2ea00fccf0 msgr2=0x7f2ea00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/1680185416 shutdown_connections 2024-04-24T23:14:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.665+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/1680185416 wait complete. 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2ea4e7b640 1 Processor -- start 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2ea4e7b640 1 -- start start 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2ea4e7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ea0101780 0x7f2ea0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2ea4e7b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0102970 0x7f2ea01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2ea4e7b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea0103330 0x7f2ea019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2ea4e7b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea010d530 con 0x7f2ea0102970 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2ea4e7b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea010d3b0 con 0x7f2ea0103330 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2ea4e7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea010d6b0 con 0x7f2ea0101780 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2e9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0102970 0x7f2ea01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2e9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0102970 0x7f2ea01980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50186/0 (socket says 172.21.15.112:50186) 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/2823594575 learned_addr learned my addr 172.21.15.112:0/2823594575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.666+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ea0101780 0x7f2ea0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/2823594575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ea0101780 msgr2=0x7f2ea0197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ea0101780 0x7f2ea0197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/2823594575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea0103330 msgr2=0x7f2ea019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea0103330 0x7f2ea019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/2823594575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea019f7b0 con 0x7f2ea0102970 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0102970 0x7f2ea01980d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400d950 tx=0x7f2e9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e8f7fe640 1 -- 172.21.15.112:0/2823594575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94017030 con 0x7f2ea0102970 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e8f7fe640 1 -- 172.21.15.112:0/2823594575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2e94004020 con 0x7f2ea0102970 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/2823594575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea019faa0 con 0x7f2ea0102970 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2e8f7fe640 1 -- 172.21.15.112:0/2823594575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94004a00 con 0x7f2ea0102970 2024-04-24T23:14:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.667+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/2823594575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea019fff0 con 0x7f2ea0102970 2024-04-24T23:14:42.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.669+0000 7f2e8f7fe640 1 -- 172.21.15.112:0/2823594575 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2e940041d0 con 0x7f2ea0102970 2024-04-24T23:14:42.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.669+0000 7f2ea7b82740 1 Processor -- start 2024-04-24T23:14:42.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2ea7b82740 1 -- start start 2024-04-24T23:14:42.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e8f7fe640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2e880a2b90 0x7f2e880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bf440 0x55f72b2bf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bfe30 0x55f72b2c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f72b222f00 con 0x55f72b2bf440 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f72b1cb750 con 0x55f72b2bfe30 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f72b21eec0 con 0x55f72b2c4fa0 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2e880a2b90 0x7f2e880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bf440 0x55f72b2bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e8f7fe640 1 -- 172.21.15.112:0/2823594575 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f2e940cee70 con 0x7f2ea0102970 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bf440 0x55f72b2bf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50192/0 (socket says 172.21.15.112:50192) 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 -- 172.21.15.112:0/1723738286 learned_addr learned my addr 172.21.15.112:0/1723738286 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bfe30 0x55f72b2c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 -- 172.21.15.112:0/1723738286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 msgr2=0x55f72b2c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 -- 172.21.15.112:0/1723738286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bfe30 msgr2=0x55f72b2c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bfe30 0x55f72b2c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 -- 172.21.15.112:0/1723738286 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72af372d0 con 0x55f72b2bf440 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bfe30 0x55f72b2c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2e880a2b90 0x7f2e880a5050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006010 tx=0x7f2e98009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.670+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bf440 0x55f72b2bf860 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000bcb0 tx=0x7f2e900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17740ba0297ed79b server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2e8cff9640 1 -- 172.21.15.112:0/1723738286 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90005020 con 0x55f72b2bf440 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2e8cff9640 1 -- 172.21.15.112:0/1723738286 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2e900047a0 con 0x55f72b2bf440 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2e8cff9640 1 -- 172.21.15.112:0/1723738286 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e900116e0 con 0x55f72b2bf440 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1723738286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bf440 msgr2=0x55f72b2bf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bf440 0x55f72b2bf860 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000bcb0 tx=0x7f2e900107a0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1723738286 shutdown_connections 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bfe30 0x55f72b2c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1723738286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bf440 0x55f72b2bf860 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1723738286 >> 172.21.15.112:0/1723738286 conn(0x55f72b2baab0 msgr2=0x55f72b2baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.671+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1723738286 shutdown_connections 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1723738286 wait complete. 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 -- start start 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 Processor -- start 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 0x55f72b2c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bfe30 0x55f72b2c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2c4fa0 0x55f72b2d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f72b2c7f50 con 0x55f72b2c4fa0 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f72b1decf0 con 0x55f72b2bf440 2024-04-24T23:14:42.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f72b2c80d0 con 0x55f72b2bfe30 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2ea7b82740 10 client.0 _ll_register_callbacks cb 0x55f72b1de6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2c4fa0 0x55f72b2d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2c4fa0 0x55f72b2d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50202/0 (socket says 172.21.15.112:50202) 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9ed9d640 1 -- 172.21.15.112:0/3888023029 learned_addr learned my addr 172.21.15.112:0/3888023029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 0x55f72b2c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bfe30 0x55f72b2c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9ed9d640 1 -- 172.21.15.112:0/3888023029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bfe30 msgr2=0x55f72b2c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bfe30 0x55f72b2c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9ed9d640 1 -- 172.21.15.112:0/3888023029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 msgr2=0x55f72b2c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 0x55f72b2c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.672+0000 7f2e9ed9d640 1 -- 172.21.15.112:0/3888023029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72b0016b0 con 0x55f72b2c4fa0 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2c4fa0 0x55f72b2d0d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2e98035000 tx=0x7f2e98043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c5aedcdda1964 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2e85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2e85ffb640 1 -- 172.21.15.112:0/3888023029 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98004d40 con 0x55f72b2c4fa0 2024-04-24T23:14:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2e85ffb640 1 -- 172.21.15.112:0/3888023029 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f2e9802fb10 con 0x55f72b2c4fa0 2024-04-24T23:14:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 1 -- 172.21.15.112:0/3888023029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e900047a0 con 0x55f72b2c4fa0 2024-04-24T23:14:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2e85ffb640 1 -- 172.21.15.112:0/3888023029 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e9805b050 con 0x55f72b2c4fa0 2024-04-24T23:14:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 10 client.15441 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 1 -- 172.21.15.112:0/3888023029 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f72b2d3dc0 con 0x55f72b2c4fa0 2024-04-24T23:14:42.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2e85ffb640 1 -- 172.21.15.112:0/3888023029 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2e900047a0 con 0x55f72b2c4fa0 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2e85ffb640 1 -- 172.21.15.112:0/3888023029 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2e98005030 con 0x55f72b2c4fa0 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 10 client.15441 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 10 client.15441 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 20 client.15441 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f72b2d6540 0x55f72b2d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 4 client.15441 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.673+0000 7f2ea7b82740 1 -- 172.21.15.112:0/3888023029 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f72b230440 con 0x55f72b2d6540 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.674+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f72b2d6540 0x55f72b2d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.674+0000 7f2e85ffb640 10 client.15441 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.674+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f72b2d6540 0x55f72b2d79b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dafdb7bf22618cfa server_cookie=1c4e38d9b5b45bb4 in_seq=0 out_seq=0 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2e85ffb640 1 -- 172.21.15.112:0/3888023029 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f72b230440 con 0x55f72b2d6540 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2e85ffb640 10 client.15441 handle_command_reply: tid=0 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 client.15441 shutdown 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 -- 172.21.15.112:0/3888023029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2c4fa0 msgr2=0x55f72b2d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2c4fa0 0x55f72b2d0d70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2e98035000 tx=0x7f2e98043000 comp rx=0 tx=0).stop 2024-04-24T23:14:42.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 -- 172.21.15.112:0/3888023029 shutdown_connections 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f72b2d6540 0x55f72b2d79b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2c4fa0 0x55f72b2d0d70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bfe30 0x55f72b2c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2e85ffb640 0 client.15441 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/3888023029 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 0x55f72b2c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 -- 172.21.15.112:0/3888023029 >> 172.21.15.112:0/3888023029 conn(0x55f72b2baab0 msgr2=0x55f72b2b9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 -- 172.21.15.112:0/3888023029 shutdown_connections 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 1 -- 172.21.15.112:0/3888023029 wait complete. 2024-04-24T23:14:42.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.675+0000 7f2ea7b82740 20 client.15441 trim_cache size 0 max 16384 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2ea7b82740 1 Processor -- start 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2ea7b82740 1 -- start start 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bf440 0x55f72b2b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b2c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f72b2d5cf0 con 0x55f72b2bfe30 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f72b2d63c0 con 0x55f72b2c4fa0 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f72b2d6a90 con 0x55f72b2bf440 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b2c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b2c06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50212/0 (socket says 172.21.15.112:50212) 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/2566617021 learned_addr learned my addr 172.21.15.112:0/2566617021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bf440 0x55f72b2b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/2566617021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bf440 msgr2=0x55f72b2b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bf440 0x55f72b2b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/2566617021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2c4fa0 msgr2=0x55f72b2c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/2566617021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72b2c0210 con 0x55f72b2bfe30 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.693+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b2c06c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2e94097cb0 tx=0x7f2e94004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f796663f8b6b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2e87fff640 1 -- 172.21.15.112:0/2566617021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9409b2e0 con 0x55f72b2bfe30 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2e87fff640 1 -- 172.21.15.112:0/2566617021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2e94095e50 con 0x55f72b2bfe30 2024-04-24T23:14:42.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2e87fff640 1 -- 172.21.15.112:0/2566617021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e940996b0 con 0x55f72b2bfe30 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 -- 172.21.15.112:0/2566617021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 msgr2=0x55f72b2c06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b2c06c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2e94097cb0 tx=0x7f2e94004540 comp rx=0 tx=0).stop 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 -- 172.21.15.112:0/2566617021 shutdown_connections 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2c4fa0 0x55f72b2c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b2c06c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/2566617021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2bf440 0x55f72b2b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 -- 172.21.15.112:0/2566617021 >> 172.21.15.112:0/2566617021 conn(0x55f72b2baab0 msgr2=0x55f72b2bae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 -- 172.21.15.112:0/2566617021 shutdown_connections 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 -- 172.21.15.112:0/2566617021 wait complete. 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 -- start start 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.694+0000 7f2ea7b82740 1 Processor -- start 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 0x55f72b338720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b338c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b33f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f72b2d7830 con 0x55f72b2bfe30 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f72b2d7160 con 0x55f72b2bf440 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f72b2d7f00 con 0x55f72b2c4fa0 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 10 client.0 _ll_register_callbacks cb 0x7f2e98031830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b338c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b338c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50216/0 (socket says 172.21.15.112:50216) 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/1224773129 learned_addr learned my addr 172.21.15.112:0/1224773129 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b33f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 msgr2=0x55f72b33f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b33f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/1224773129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 msgr2=0x55f72b338720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 0x55f72b338720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 -- 172.21.15.112:0/1224773129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72b33fb90 con 0x55f72b2bfe30 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9ed9d640 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b33f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e9dd9b640 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b338c60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2e94004770 tx=0x7f2e9409f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad4a3f67160e2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e85ffb640 1 -- 172.21.15.112:0/1224773129 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e940a3f50 con 0x55f72b2bfe30 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e85ffb640 1 -- 172.21.15.112:0/1224773129 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f2e940b4210 con 0x55f72b2bfe30 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1224773129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e94095e50 con 0x55f72b2bfe30 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2e85ffb640 1 -- 172.21.15.112:0/1224773129 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2e94097950 con 0x55f72b2bfe30 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 10 client.15447 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.695+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1224773129 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f72b340da0 con 0x55f72b2bfe30 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2e85ffb640 1 -- 172.21.15.112:0/1224773129 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2e94095e50 con 0x55f72b2bfe30 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2e85ffb640 1 -- 172.21.15.112:0/1224773129 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2e940bb0b0 con 0x55f72b2bfe30 2024-04-24T23:14:42.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2ea7b82740 10 client.15447 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2ea7b82740 10 client.15447 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2ea7b82740 20 client.15447 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f72b345570 0x55f72b3419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2ea7b82740 4 client.15447 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1224773129 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "status"}) -- 0x55f72b22a790 con 0x55f72b345570 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f72b345570 0x55f72b3419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2e85ffb640 10 client.15447 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.696+0000 7f2e9e59c640 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f72b345570 0x55f72b3419b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb78d3d3a555d6d6 server_cookie=7615c7e54797fa36 in_seq=0 out_seq=0 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2e85ffb640 1 -- 172.21.15.112:0/1224773129 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55f72b22a790 con 0x55f72b345570 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2e85ffb640 10 client.15447 handle_command_reply: tid=0 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 client.15447 shutdown 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1224773129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 msgr2=0x55f72b338c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b338c60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2e94004770 tx=0x7f2e9409f710 comp rx=0 tx=0).stop 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1224773129 shutdown_connections 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f72b345570 0x55f72b3419b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f72b2c4fa0 0x55f72b33f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f72b2bfe30 0x55f72b338c60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2e85ffb640 0 client.15447 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 --2- 172.21.15.112:0/1224773129 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f72b2bf440 0x55f72b338720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1224773129 >> 172.21.15.112:0/1224773129 conn(0x55f72b2baab0 msgr2=0x55f72b2c06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1224773129 shutdown_connections 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 1 -- 172.21.15.112:0/1224773129 wait complete. 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea7b82740 20 client.15447 trim_cache size 0 max 16384 2024-04-24T23:14:42.707 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 4, 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24424, 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 107.82529828, 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 172.761895834 2024-04-24T23:14:42.708 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2e880a2b90 msgr2=0x7f2e880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2e880a2b90 0x7f2e880a5050 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2e98006010 tx=0x7f2e98009800 comp rx=0 tx=0).stop 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0102970 msgr2=0x7f2ea01980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0102970 0x7f2ea01980d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400d950 tx=0x7f2e9400de10 comp rx=0 tx=0).stop 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/2823594575 shutdown_connections 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2e880a2b90 0x7f2e880a5050 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ea0103330 0x7f2ea019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ea0102970 0x7f2ea01980d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 --2- 172.21.15.112:0/2823594575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ea0101780 0x7f2ea0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/2823594575 >> 172.21.15.112:0/2823594575 conn(0x7f2ea00fccf0 msgr2=0x7f2ea0105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/2823594575 shutdown_connections 2024-04-24T23:14:42.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.697+0000 7f2ea4e7b640 1 -- 172.21.15.112:0/2823594575 wait complete. 2024-04-24T23:14:42.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-24T23:14:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 -- 172.21.15.112:0/1608448881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 msgr2=0x7f652c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 --2- 172.21.15.112:0/1608448881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 0x7f652c101bc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6520009900 tx=0x7f652002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 -- 172.21.15.112:0/1608448881 shutdown_connections 2024-04-24T23:14:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 --2- 172.21.15.112:0/1608448881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f652c102100 0x7f652c108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 --2- 172.21.15.112:0/1608448881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 0x7f652c101bc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 --2- 172.21.15.112:0/1608448881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c100550 0x7f652c100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 -- 172.21.15.112:0/1608448881 >> 172.21.15.112:0/1608448881 conn(0x7f652c0fbae0 msgr2=0x7f652c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 -- 172.21.15.112:0/1608448881 shutdown_connections 2024-04-24T23:14:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 -- 172.21.15.112:0/1608448881 wait complete. 2024-04-24T23:14:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.811+0000 7f6530f9e640 1 Processor -- start 2024-04-24T23:14:42.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f6530f9e640 1 -- start start 2024-04-24T23:14:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f6530f9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f652c100550 0x7f652c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f6530f9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 0x7f652c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f6530f9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c102100 0x7f652c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f6530f9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f652c10c300 con 0x7f652c101740 2024-04-24T23:14:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f6530f9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f652c10c180 con 0x7f652c100550 2024-04-24T23:14:42.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f6530f9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f652c10c480 con 0x7f652c102100 2024-04-24T23:14:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 0x7f652c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 0x7f652c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50234/0 (socket says 172.21.15.112:50234) 2024-04-24T23:14:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652affd640 1 -- 172.21.15.112:0/2971625913 learned_addr learned my addr 172.21.15.112:0/2971625913 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652bfff640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c102100 0x7f652c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652b7fe640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f652c100550 0x7f652c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652affd640 1 -- 172.21.15.112:0/2971625913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c102100 msgr2=0x7f652c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652affd640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c102100 0x7f652c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652affd640 1 -- 172.21.15.112:0/2971625913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f652c100550 msgr2=0x7f652c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652affd640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f652c100550 0x7f652c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652affd640 1 -- 172.21.15.112:0/2971625913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c19f610 con 0x7f652c101740 2024-04-24T23:14:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.812+0000 7f652bfff640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c102100 0x7f652c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.813+0000 7f652affd640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 0x7f652c19c6e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6520009a20 tx=0x7f6520004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.813+0000 7f6528ff9640 1 -- 172.21.15.112:0/2971625913 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652003f030 con 0x7f652c101740 2024-04-24T23:14:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.813+0000 7f6530f9e640 1 -- 172.21.15.112:0/2971625913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652c19f840 con 0x7f652c101740 2024-04-24T23:14:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.813+0000 7f6528ff9640 1 -- 172.21.15.112:0/2971625913 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f6520003d00 con 0x7f652c101740 2024-04-24T23:14:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.813+0000 7f6528ff9640 1 -- 172.21.15.112:0/2971625913 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520035070 con 0x7f652c101740 2024-04-24T23:14:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.813+0000 7f6530f9e640 1 -- 172.21.15.112:0/2971625913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652c19fd80 con 0x7f652c101740 2024-04-24T23:14:42.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6528ff9640 1 -- 172.21.15.112:0/2971625913 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f6520042030 con 0x7f652c101740 2024-04-24T23:14:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6534fee740 1 Processor -- start 2024-04-24T23:14:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6534fee740 1 -- start start 2024-04-24T23:14:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6534fee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 0x55edd6441a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6528ff9640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f65140a2b90 0x7f65140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6534fee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6442030 0x55edd6446b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6534fee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6449630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6534fee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55edd63a4f00 con 0x55edd6442030 2024-04-24T23:14:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6534fee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55edd634d750 con 0x55edd64471a0 2024-04-24T23:14:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f6534fee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55edd63a0ec0 con 0x55edd6441640 2024-04-24T23:14:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f652affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6442030 0x55edd6446b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.815+0000 7f652b7fe640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f65140a2b90 0x7f65140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652bfff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6449630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6442030 0x55edd6446b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50242/0 (socket says 172.21.15.112:50242) 2024-04-24T23:14:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652affd640 1 -- 172.21.15.112:0/3356849468 learned_addr learned my addr 172.21.15.112:0/3356849468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f6528ff9640 1 -- 172.21.15.112:0/2971625913 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f65200efba0 con 0x7f652c101740 2024-04-24T23:14:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652bfff640 1 --2- 172.21.15.112:0/3356849468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 0x55edd6441a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652affd640 1 -- 172.21.15.112:0/3356849468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 msgr2=0x55edd6441a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652affd640 1 --2- 172.21.15.112:0/3356849468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 0x55edd6441a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652b7fe640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f65140a2b90 0x7f65140a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f651c0058c0 tx=0x7f651c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652affd640 1 -- 172.21.15.112:0/3356849468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 msgr2=0x55edd6449630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652affd640 1 --2- 172.21.15.112:0/3356849468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6449630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652affd640 1 -- 172.21.15.112:0/3356849468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd60b92d0 con 0x55edd6442030 2024-04-24T23:14:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f652affd640 1 --2- 172.21.15.112:0/3356849468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6442030 0x55edd6446b30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f65200ed2b0 tx=0x7f652002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca73a25ab1849dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f6511ffb640 1 -- 172.21.15.112:0/3356849468 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520049070 con 0x55edd6442030 2024-04-24T23:14:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f6511ffb640 1 -- 172.21.15.112:0/3356849468 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f652003be30 con 0x55edd6442030 2024-04-24T23:14:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.816+0000 7f6511ffb640 1 -- 172.21.15.112:0/3356849468 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200c43f0 con 0x55edd6442030 2024-04-24T23:14:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 -- 172.21.15.112:0/3356849468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6442030 msgr2=0x55edd6446b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 --2- 172.21.15.112:0/3356849468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6442030 0x55edd6446b30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f65200ed2b0 tx=0x7f652002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 -- 172.21.15.112:0/3356849468 shutdown_connections 2024-04-24T23:14:42.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 --2- 172.21.15.112:0/3356849468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6449630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 --2- 172.21.15.112:0/3356849468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6442030 0x55edd6446b30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 --2- 172.21.15.112:0/3356849468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 0x55edd6441a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 -- 172.21.15.112:0/3356849468 >> 172.21.15.112:0/3356849468 conn(0x55edd643ccb0 msgr2=0x55edd643d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 -- 172.21.15.112:0/3356849468 shutdown_connections 2024-04-24T23:14:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 -- 172.21.15.112:0/3356849468 wait complete. 2024-04-24T23:14:42.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.817+0000 7f6534fee740 1 -- start start 2024-04-24T23:14:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f6534fee740 1 Processor -- start 2024-04-24T23:14:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f6534fee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 0x55edd644b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f6534fee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd644bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f6534fee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd64471a0 0x55edd6452d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f6534fee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55edd6449f10 con 0x55edd64471a0 2024-04-24T23:14:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f6534fee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55edd6360cf0 con 0x55edd6442030 2024-04-24T23:14:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f6534fee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55edd644a090 con 0x55edd6441640 2024-04-24T23:14:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f6534fee740 10 client.0 _ll_register_callbacks cb 0x55edd63606e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd64471a0 0x55edd6452d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd64471a0 0x55edd6452d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50256/0 (socket says 172.21.15.112:50256) 2024-04-24T23:14:42.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 -- 172.21.15.112:0/1205919263 learned_addr learned my addr 172.21.15.112:0/1205919263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652b7fe640 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 0x55edd644b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652affd640 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd644bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 -- 172.21.15.112:0/1205919263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 msgr2=0x55edd644b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 0x55edd644b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 -- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 msgr2=0x55edd644bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd644bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 -- 172.21.15.112:0/1205919263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd61836b0 con 0x55edd64471a0 2024-04-24T23:14:42.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652affd640 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd644bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f652bfff640 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd64471a0 0x55edd6452d50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6524009bb0 tx=0x7f65240128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb1422d541b2d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.818+0000 7f650affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:42.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f650affd640 1 -- 172.21.15.112:0/1205919263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524003970 con 0x55edd64471a0 2024-04-24T23:14:42.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f650affd640 1 -- 172.21.15.112:0/1205919263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f6524003b10 con 0x55edd64471a0 2024-04-24T23:14:42.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 1 -- 172.21.15.112:0/1205919263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f652003be30 con 0x55edd64471a0 2024-04-24T23:14:42.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f650affd640 1 -- 172.21.15.112:0/1205919263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6524003d10 con 0x55edd64471a0 2024-04-24T23:14:42.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 10 client.15459 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:42.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 1 -- 172.21.15.112:0/1205919263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55edd6455da0 con 0x55edd64471a0 2024-04-24T23:14:42.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f650affd640 1 -- 172.21.15.112:0/1205919263 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f652003be30 con 0x55edd64471a0 2024-04-24T23:14:42.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f650affd640 1 -- 172.21.15.112:0/1205919263 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f652401f7e0 con 0x55edd64471a0 2024-04-24T23:14:42.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 10 client.15459 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:42.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 10 client.15459 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-24T23:14:42.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 20 client.15459 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:42.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55edd64583f0 0x55edd6459860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 4 client.15459 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.819+0000 7f6534fee740 1 -- 172.21.15.112:0/1205919263 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edd63b2440 con 0x55edd64583f0 2024-04-24T23:14:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.820+0000 7f652b7fe640 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55edd64583f0 0x55edd6459860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.820+0000 7f650affd640 10 client.15459 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:14:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.820+0000 7f652b7fe640 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55edd64583f0 0x55edd6459860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=33a78108a5735e3d server_cookie=bf45a4726d4af1fc in_seq=0 out_seq=0 2024-04-24T23:14:42.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f650affd640 1 -- 172.21.15.112:0/1205919263 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55edd63b2440 con 0x55edd64583f0 2024-04-24T23:14:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f650affd640 10 client.15459 handle_command_reply: tid=0 2024-04-24T23:14:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 client.15459 shutdown 2024-04-24T23:14:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 -- 172.21.15.112:0/1205919263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd64471a0 msgr2=0x55edd6452d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd64471a0 0x55edd6452d50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6524009bb0 tx=0x7f65240128e0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 -- 172.21.15.112:0/1205919263 shutdown_connections 2024-04-24T23:14:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55edd64583f0 0x55edd6459860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd64471a0 0x55edd6452d50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd644bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 --2- 172.21.15.112:0/1205919263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6441640 0x55edd644b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f650affd640 0 client.15459 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:14:42.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 -- 172.21.15.112:0/1205919263 >> 172.21.15.112:0/1205919263 conn(0x55edd643ccb0 msgr2=0x55edd643be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 -- 172.21.15.112:0/1205919263 shutdown_connections 2024-04-24T23:14:42.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 1 -- 172.21.15.112:0/1205919263 wait complete. 2024-04-24T23:14:42.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.821+0000 7f6534fee740 20 client.15459 trim_cache size 0 max 16384 2024-04-24T23:14:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f6534fee740 1 Processor -- start 2024-04-24T23:14:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f6534fee740 1 -- start start 2024-04-24T23:14:42.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f6534fee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f6534fee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd6443b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f6534fee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd64471a0 0x55edd6452d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f6534fee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55edd6457ba0 con 0x55edd6441640 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f6534fee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55edd6458270 con 0x55edd6442030 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f6534fee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55edd6458940 con 0x55edd64471a0 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f652b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f652b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50258/0 (socket says 172.21.15.112:50258) 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f652b7fe640 1 -- 172.21.15.112:0/3604052826 learned_addr learned my addr 172.21.15.112:0/3604052826 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.839+0000 7f652affd640 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd6443b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652b7fe640 1 -- 172.21.15.112:0/3604052826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd64471a0 msgr2=0x55edd6452d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652bfff640 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd64471a0 0x55edd6452d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652b7fe640 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd64471a0 0x55edd6452d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652b7fe640 1 -- 172.21.15.112:0/3604052826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 msgr2=0x55edd6443b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652b7fe640 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd6443b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652b7fe640 1 -- 172.21.15.112:0/3604052826 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edd6446820 con 0x55edd6441640 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652bfff640 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd64471a0 0x55edd6452d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652affd640 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd6443b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f652b7fe640 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64435d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f652c1015a0 tx=0x7f651c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a474ec995ee3f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f65117fa640 1 -- 172.21.15.112:0/3604052826 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c02c070 con 0x55edd6441640 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f65117fa640 1 -- 172.21.15.112:0/3604052826 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f651c01ee40 con 0x55edd6441640 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f65117fa640 1 -- 172.21.15.112:0/3604052826 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c01d440 con 0x55edd6441640 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f6534fee740 1 -- 172.21.15.112:0/3604052826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 msgr2=0x55edd64435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f6534fee740 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64435d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f652c1015a0 tx=0x7f651c014000 comp rx=0 tx=0).stop 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f6534fee740 1 -- 172.21.15.112:0/3604052826 shutdown_connections 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f6534fee740 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd64471a0 0x55edd6452d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f6534fee740 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd6442030 0x55edd6443b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f6534fee740 1 --2- 172.21.15.112:0/3604052826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64435d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.840+0000 7f6534fee740 1 -- 172.21.15.112:0/3604052826 >> 172.21.15.112:0/3604052826 conn(0x55edd643ccb0 msgr2=0x55edd643d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 -- 172.21.15.112:0/3604052826 shutdown_connections 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 -- 172.21.15.112:0/3604052826 wait complete. 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 -- start start 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 Processor -- start 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6442030 0x55edd6445b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6451300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55edd64596e0 con 0x55edd6441640 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55edd6459010 con 0x55edd64471a0 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55edd6459db0 con 0x55edd6442030 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f6534fee740 10 client.0 _ll_register_callbacks cb 0x7f6524014c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f652b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f652b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50270/0 (socket says 172.21.15.112:50270) 2024-04-24T23:14:42.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f652b7fe640 1 -- 172.21.15.112:0/1139957464 learned_addr learned my addr 172.21.15.112:0/1139957464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f652affd640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6442030 0x55edd6445b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.841+0000 7f652bfff640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6451300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f652b7fe640 1 -- 172.21.15.112:0/1139957464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6442030 msgr2=0x55edd6445b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f652b7fe640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6442030 0x55edd6445b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f652b7fe640 1 -- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 msgr2=0x55edd6451300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f652b7fe640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6451300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f652b7fe640 1 -- 172.21.15.112:0/1139957464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edd64518f0 con 0x55edd6441640 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f652affd640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6442030 0x55edd6445b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f652bfff640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6451300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f652b7fe640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64455f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f651c01ec90 tx=0x7f651c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74a9348a4efb74e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f650affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f650affd640 1 -- 172.21.15.112:0/1139957464 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c02c040 con 0x55edd6441640 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f650affd640 1 -- 172.21.15.112:0/1139957464 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f651c0275a0 con 0x55edd6441640 2024-04-24T23:14:42.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 1 -- 172.21.15.112:0/1139957464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55edd63ac790 con 0x55edd6441640 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f650affd640 1 -- 172.21.15.112:0/1139957464 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f651c03f050 con 0x55edd6441640 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 10 client.15465 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 1 -- 172.21.15.112:0/1139957464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55edd6452b60 con 0x55edd6441640 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f650affd640 1 -- 172.21.15.112:0/1139957464 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55edd63ac790 con 0x55edd6441640 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f650affd640 1 -- 172.21.15.112:0/1139957464 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f651c0049a0 con 0x55edd6441640 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 10 client.15465 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 10 client.15465 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 20 client.15465 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55edd64c2560 0x55edd64c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 4 client.15465 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.842+0000 7f6534fee740 1 -- 172.21.15.112:0/1139957464 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "status"}) -- 0x55edd632ab80 con 0x55edd64c2560 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.843+0000 7f652affd640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55edd64c2560 0x55edd64c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.843+0000 7f650affd640 10 client.15465 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:14:42.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.843+0000 7f652affd640 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55edd64c2560 0x55edd64c1270 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=485a896ebe809641 server_cookie=7dc439a3c4395ea8 in_seq=0 out_seq=0 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.843+0000 7f650affd640 1 -- 172.21.15.112:0/1139957464 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55edd632ab80 con 0x55edd64c2560 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.843+0000 7f650affd640 10 client.15465 handle_command_reply: tid=0 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.843+0000 7f6534fee740 1 client.15465 shutdown 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 -- 172.21.15.112:0/1139957464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 msgr2=0x55edd64455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64455f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f651c01ec90 tx=0x7f651c010da0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 -- 172.21.15.112:0/1139957464 shutdown_connections 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55edd64c2560 0x55edd64c1270 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55edd64471a0 0x55edd6451300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55edd6442030 0x55edd6445b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f650affd640 0 client.15465 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 --2- 172.21.15.112:0/1139957464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55edd6441640 0x55edd64455f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 -- 172.21.15.112:0/1139957464 >> 172.21.15.112:0/1139957464 conn(0x55edd643ccb0 msgr2=0x55edd64435d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 -- 172.21.15.112:0/1139957464 shutdown_connections 2024-04-24T23:14:42.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 1 -- 172.21.15.112:0/1139957464 wait complete. 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6534fee740 20 client.15465 trim_cache size 0 max 16384 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 3, 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24425, 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 108.979755263, 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:42.868 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-24T23:14:42.869 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:42.869 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 169.98290635999999 2024-04-24T23:14:42.869 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 -- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f65140a2b90 msgr2=0x7f65140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f65140a2b90 0x7f65140a5050 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f651c0058c0 tx=0x7f651c005850 comp rx=0 tx=0).stop 2024-04-24T23:14:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 -- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 msgr2=0x7f652c19c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 0x7f652c19c6e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6520009a20 tx=0x7f6520004430 comp rx=0 tx=0).stop 2024-04-24T23:14:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 -- 172.21.15.112:0/2971625913 shutdown_connections 2024-04-24T23:14:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f65140a2b90 0x7f65140a5050 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f652c102100 0x7f652c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f652c101740 0x7f652c19c6e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 --2- 172.21.15.112:0/2971625913 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f652c100550 0x7f652c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 -- 172.21.15.112:0/2971625913 >> 172.21.15.112:0/2971625913 conn(0x7f652c0fbae0 msgr2=0x7f652c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 -- 172.21.15.112:0/2971625913 shutdown_connections 2024-04-24T23:14:42.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.844+0000 7f6530f9e640 1 -- 172.21.15.112:0/2971625913 wait complete. 2024-04-24T23:14:42.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-24T23:14:42.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 -- 172.21.15.112:0/938649367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c100ef0 msgr2=0x7fbb1c101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 --2- 172.21.15.112:0/938649367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c101370 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009900 tx=0x7fbb0c02a410 comp rx=0 tx=0).stop 2024-04-24T23:14:42.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 -- 172.21.15.112:0/938649367 shutdown_connections 2024-04-24T23:14:42.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 --2- 172.21.15.112:0/938649367 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb1c1018b0 0x7fbb1c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 --2- 172.21.15.112:0/938649367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c101370 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 --2- 172.21.15.112:0/938649367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1c0ffd00 0x7fbb1c100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 -- 172.21.15.112:0/938649367 >> 172.21.15.112:0/938649367 conn(0x7fbb1c0fb270 msgr2=0x7fbb1c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 -- 172.21.15.112:0/938649367 shutdown_connections 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.970+0000 7fbb2299e640 1 -- 172.21.15.112:0/938649367 wait complete. 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2299e640 1 Processor -- start 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2299e640 1 -- start start 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2299e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c0ffd00 0x7fbb1c1096d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2299e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2299e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb1c1018b0 0x7fbb1c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2299e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c10c3b0 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2299e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c10c230 con 0x7fbb1c1018b0 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2299e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c10c530 con 0x7fbb1c100ef0 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c0ffd00 0x7fbb1c1096d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c0ffd00 0x7fbb1c1096d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50292/0 (socket says 172.21.15.112:50292) 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 -- 172.21.15.112:0/3869173861 learned_addr learned my addr 172.21.15.112:0/3869173861 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2119b640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2219d640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb1c1018b0 0x7fbb1c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 -- 172.21.15.112:0/3869173861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1c100ef0 msgr2=0x7fbb1c19c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c19c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 -- 172.21.15.112:0/3869173861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb1c1018b0 msgr2=0x7fbb1c19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb1c1018b0 0x7fbb1c19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 -- 172.21.15.112:0/3869173861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c19f690 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2119b640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2219d640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb1c1018b0 0x7fbb1c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb2199c640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c0ffd00 0x7fbb1c1096d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400b450 tx=0x7fbb1400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.971+0000 7fbb12ffd640 1 -- 172.21.15.112:0/3869173861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14015020 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.972+0000 7fbb2299e640 1 -- 172.21.15.112:0/3869173861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c19f980 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.972+0000 7fbb12ffd640 1 -- 172.21.15.112:0/3869173861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb14011d90 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.972+0000 7fbb12ffd640 1 -- 172.21.15.112:0/3869173861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400f340 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.972+0000 7fbb2299e640 1 -- 172.21.15.112:0/3869173861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c19ff00 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.974+0000 7fbb12ffd640 1 -- 172.21.15.112:0/3869173861 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbb14018020 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.974+0000 7fbb269ed740 1 Processor -- start 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb269ed740 1 -- start start 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 0x55c704d5ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d60b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d63630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb269ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c704cbef00 con 0x55c704d5c030 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb269ed740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c704c67750 con 0x55c704d5b640 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb269ed740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c704cbaec0 con 0x55c704d611a0 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb12ffd640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbb080a2b90 0x7fbb080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d60b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2219d640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbb080a2b90 0x7fbb080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d60b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50298/0 (socket says 172.21.15.112:50298) 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2119b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d63630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 -- 172.21.15.112:0/1227514267 learned_addr learned my addr 172.21.15.112:0/1227514267 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb12ffd640 1 -- 172.21.15.112:0/3869173861 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbb140cefe0 con 0x7fbb1c0ffd00 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2219d640 1 --2- 172.21.15.112:0/1227514267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 0x55c704d5ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 -- 172.21.15.112:0/1227514267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 msgr2=0x55c704d63630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 --2- 172.21.15.112:0/1227514267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d63630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 -- 172.21.15.112:0/1227514267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 msgr2=0x55c704d5ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 --2- 172.21.15.112:0/1227514267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 0x55c704d5ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 -- 172.21.15.112:0/1227514267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7049d32d0 con 0x55c704d5c030 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2199c640 1 --2- 172.21.15.112:0/1227514267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d60b30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbb14010a10 tx=0x7fbb1406e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622910a4e364a9fb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.975+0000 7fbb2219d640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbb080a2b90 0x7fbb080a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005fd0 tx=0x7fbb1800a9c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb07fff640 1 -- 172.21.15.112:0/1227514267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1409bb10 con 0x55c704d5c030 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb07fff640 1 -- 172.21.15.112:0/1227514267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb140a4080 con 0x55c704d5c030 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb07fff640 1 -- 172.21.15.112:0/1227514267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb140cc7c0 con 0x55c704d5c030 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb269ed740 1 -- 172.21.15.112:0/1227514267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 msgr2=0x55c704d60b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1227514267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d60b30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbb14010a10 tx=0x7fbb1406e4d0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb269ed740 1 -- 172.21.15.112:0/1227514267 shutdown_connections 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1227514267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d63630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1227514267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d60b30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1227514267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 0x55c704d5ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.976+0000 7fbb269ed740 1 -- 172.21.15.112:0/1227514267 >> 172.21.15.112:0/1227514267 conn(0x55c704d56cb0 msgr2=0x55c704d570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 -- 172.21.15.112:0/1227514267 shutdown_connections 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 -- 172.21.15.112:0/1227514267 wait complete. 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 -- start start 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 Processor -- start 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5b640 0x55c704d65a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d65f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c704d64150 con 0x55c704d5b640 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c704c7acf0 con 0x55c704d5c030 2024-04-24T23:14:42.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c704d642d0 con 0x55c704d611a0 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb269ed740 10 client.0 _ll_register_callbacks cb 0x55c704c7a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb2199c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5b640 0x55c704d65a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb2199c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5b640 0x55c704d65a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50308/0 (socket says 172.21.15.112:50308) 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb2199c640 1 -- 172.21.15.112:0/1062428936 learned_addr learned my addr 172.21.15.112:0/1062428936 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.977+0000 7fbb2119b640 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d65f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb2219d640 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb2199c640 1 -- 172.21.15.112:0/1062428936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 msgr2=0x55c704d6cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb2199c640 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb2199c640 1 -- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 msgr2=0x55c704d65f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb2199c640 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d65f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb2199c640 1 -- 172.21.15.112:0/1062428936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c704a9d6b0 con 0x55c704d5b640 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb2119b640 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d65f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb2199c640 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5b640 0x55c704d65a40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400be10 tx=0x7fbb14066630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985aa0ff6a97b02d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:42.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb057fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb057fa640 1 -- 172.21.15.112:0/1062428936 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb140b3070 con 0x55c704d5b640 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb057fa640 1 -- 172.21.15.112:0/1062428936 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbb140b4260 con 0x55c704d5b640 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 1 -- 172.21.15.112:0/1062428936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb140a4080 con 0x55c704d5b640 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb057fa640 1 -- 172.21.15.112:0/1062428936 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbb1409b410 con 0x55c704d5b640 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 10 client.15477 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 1 -- 172.21.15.112:0/1062428936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c704d6ffa0 con 0x55c704d5b640 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb057fa640 1 -- 172.21.15.112:0/1062428936 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb140a4080 con 0x55c704d5b640 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb057fa640 1 -- 172.21.15.112:0/1062428936 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbb140ccb70 con 0x55c704d5b640 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 10 client.15477 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 20 client.15477 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x55c704d712f0 0x55c704d734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 4 client.15477 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.978+0000 7fbb269ed740 1 -- 172.21.15.112:0/1062428936 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c704ccc440 con 0x55c704d712f0 2024-04-24T23:14:42.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.979+0000 7fbb2119b640 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x55c704d712f0 0x55c704d734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.979+0000 7fbb2119b640 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x55c704d712f0 0x55c704d734d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e150243f23cdac2 server_cookie=ba504264ed2f294 in_seq=0 out_seq=0 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.979+0000 7fbb057fa640 10 client.15477 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb057fa640 1 -- 172.21.15.112:0/1062428936 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c704ccc440 con 0x55c704d712f0 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb057fa640 10 client.15477 handle_command_reply: tid=0 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 client.15477 shutdown 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 -- 172.21.15.112:0/1062428936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5b640 msgr2=0x55c704d65a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5b640 0x55c704d65a40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400be10 tx=0x7fbb14066630 comp rx=0 tx=0).stop 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 -- 172.21.15.112:0/1062428936 shutdown_connections 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x55c704d712f0 0x55c704d734d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb057fa640 0 client.15477 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d65f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 --2- 172.21.15.112:0/1062428936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5b640 0x55c704d65a40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:42.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 -- 172.21.15.112:0/1062428936 >> 172.21.15.112:0/1062428936 conn(0x55c704d56cb0 msgr2=0x55c704d55e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:42.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 -- 172.21.15.112:0/1062428936 shutdown_connections 2024-04-24T23:14:42.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 1 -- 172.21.15.112:0/1062428936 wait complete. 2024-04-24T23:14:42.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.980+0000 7fbb269ed740 20 client.15477 trim_cache size 0 max 16384 2024-04-24T23:14:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.998+0000 7fbb269ed740 1 Processor -- start 2024-04-24T23:14:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.998+0000 7fbb269ed740 1 -- start start 2024-04-24T23:14:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.998+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d5b640 0x55c704d5caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.998+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d6cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.998+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d611a0 0x55c704d65650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.998+0000 7fbb269ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c704d71810 con 0x55c704d611a0 2024-04-24T23:14:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.998+0000 7fbb269ed740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c704d71ee0 con 0x55c704d5c030 2024-04-24T23:14:43.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.998+0000 7fbb269ed740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c704d725b0 con 0x55c704d5b640 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d611a0 0x55c704d65650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d611a0 0x55c704d65650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50316/0 (socket says 172.21.15.112:50316) 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 -- 172.21.15.112:0/403644848 learned_addr learned my addr 172.21.15.112:0/403644848 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2199c640 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d5b640 0x55c704d5caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2119b640 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d6cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 -- 172.21.15.112:0/403644848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d5b640 msgr2=0x55c704d5caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d5b640 0x55c704d5caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 -- 172.21.15.112:0/403644848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 msgr2=0x55c704d6cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d6cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 -- 172.21.15.112:0/403644848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c704d5c410 con 0x55c704d611a0 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2119b640 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d6cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb2219d640 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d611a0 0x55c704d65650 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800adb0 tx=0x7fbb18017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6b485e670eba6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb077fe640 1 -- 172.21.15.112:0/403644848 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18012650 con 0x55c704d611a0 2024-04-24T23:14:43.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb077fe640 1 -- 172.21.15.112:0/403644848 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb18023070 con 0x55c704d611a0 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:42.999+0000 7fbb077fe640 1 -- 172.21.15.112:0/403644848 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1801ec80 con 0x55c704d611a0 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 -- 172.21.15.112:0/403644848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d611a0 msgr2=0x55c704d65650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d611a0 0x55c704d65650 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800adb0 tx=0x7fbb18017000 comp rx=0 tx=0).stop 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 -- 172.21.15.112:0/403644848 shutdown_connections 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d611a0 0x55c704d65650 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5c030 0x55c704d6cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 --2- 172.21.15.112:0/403644848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d5b640 0x55c704d5caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 -- 172.21.15.112:0/403644848 >> 172.21.15.112:0/403644848 conn(0x55c704d56cb0 msgr2=0x55c704d5d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 -- 172.21.15.112:0/403644848 shutdown_connections 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 -- 172.21.15.112:0/403644848 wait complete. 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 -- start start 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.000+0000 7fbb269ed740 1 Processor -- start 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 0x55c704d68900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d68e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb269ed740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb269ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c704d73350 con 0x55c704d5c030 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb269ed740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c704d72c80 con 0x55c704d5b640 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb269ed740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c704d73a20 con 0x55c704d611a0 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb269ed740 10 client.0 _ll_register_callbacks cb 0x7fbb1407a0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d68e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d68e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50326/0 (socket says 172.21.15.112:50326) 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 -- 172.21.15.112:0/3537718616 learned_addr learned my addr 172.21.15.112:0/3537718616 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2199c640 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 0x55c704d68900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2219d640 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 -- 172.21.15.112:0/3537718616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 msgr2=0x55c704d6be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 -- 172.21.15.112:0/3537718616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 msgr2=0x55c704d68900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 0x55c704d68900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 -- 172.21.15.112:0/3537718616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c704d6c450 con 0x55c704d5c030 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2219d640 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb2119b640 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d68e40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c032fd0 tx=0x7fbb0c0041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9704f93f338e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb057fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb057fa640 1 -- 172.21.15.112:0/3537718616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c035040 con 0x55c704d5c030 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb057fa640 1 -- 172.21.15.112:0/3537718616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbb0c043d50 con 0x55c704d5c030 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.001+0000 7fbb269ed740 1 -- 172.21.15.112:0/3537718616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb18023070 con 0x55c704d5c030 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb057fa640 1 -- 172.21.15.112:0/3537718616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c052050 con 0x55c704d5c030 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb269ed740 10 client.15483 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb269ed740 1 -- 172.21.15.112:0/3537718616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c704dd8250 con 0x55c704d5c030 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb057fa640 1 -- 172.21.15.112:0/3537718616 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbb18023070 con 0x55c704d5c030 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb057fa640 1 -- 172.21.15.112:0/3537718616 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbb0c043250 con 0x55c704d5c030 2024-04-24T23:14:43.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb269ed740 10 client.15483 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb269ed740 10 client.15483 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb269ed740 20 client.15483 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb269ed740 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x55c704ddcef0 0x55c704ddbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb269ed740 4 client.15483 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb269ed740 1 -- 172.21.15.112:0/3537718616 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "status"}) -- 0x55c704cc6790 con 0x55c704ddcef0 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.002+0000 7fbb2199c640 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x55c704ddcef0 0x55c704ddbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb057fa640 10 client.15483 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb2199c640 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x55c704ddcef0 0x55c704ddbc00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29f094623b7c493 server_cookie=6b56511bb2763acb in_seq=0 out_seq=0 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb057fa640 1 -- 172.21.15.112:0/3537718616 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c704cc6790 con 0x55c704ddcef0 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb057fa640 10 client.15483 handle_command_reply: tid=0 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 client.15483 shutdown 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 -- 172.21.15.112:0/3537718616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 msgr2=0x55c704d68e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d68e40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c032fd0 tx=0x7fbb0c0041a0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 -- 172.21.15.112:0/3537718616 shutdown_connections 2024-04-24T23:14:43.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x55c704ddcef0 0x55c704ddbc00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c704d611a0 0x55c704d6be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c704d5c030 0x55c704d68e40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb057fa640 0 client.15483 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 --2- 172.21.15.112:0/3537718616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c704d5b640 0x55c704d68900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 -- 172.21.15.112:0/3537718616 >> 172.21.15.112:0/3537718616 conn(0x55c704d56cb0 msgr2=0x55c704d65650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 -- 172.21.15.112:0/3537718616 shutdown_connections 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 1 -- 172.21.15.112:0/3537718616 wait complete. 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.003+0000 7fbb269ed740 20 client.15483 trim_cache size 0 max 16384 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:43.012 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24482, 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 110.11416681599999, 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 153.055117166 2024-04-24T23:14:43.013 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:43.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 -- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbb080a2b90 msgr2=0x7fbb080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbb080a2b90 0x7fbb080a5050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005fd0 tx=0x7fbb1800a9c0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 -- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c0ffd00 msgr2=0x7fbb1c1096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c0ffd00 0x7fbb1c1096d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400b450 tx=0x7fbb1400b910 comp rx=0 tx=0).stop 2024-04-24T23:14:43.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 -- 172.21.15.112:0/3869173861 shutdown_connections 2024-04-24T23:14:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbb080a2b90 0x7fbb080a5050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb1c1018b0 0x7fbb1c19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb1c100ef0 0x7fbb1c19c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 --2- 172.21.15.112:0/3869173861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbb1c0ffd00 0x7fbb1c1096d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 -- 172.21.15.112:0/3869173861 >> 172.21.15.112:0/3869173861 conn(0x7fbb1c0fb270 msgr2=0x7fbb1c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 -- 172.21.15.112:0/3869173861 shutdown_connections 2024-04-24T23:14:43.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.004+0000 7fbb2299e640 1 -- 172.21.15.112:0/3869173861 wait complete. 2024-04-24T23:14:43.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-24T23:14:43.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.113+0000 7f4538abd640 1 -- 172.21.15.112:0/1919085838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4534101890 msgr2=0x7f4534101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.113+0000 7f4538abd640 1 --2- 172.21.15.112:0/1919085838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4534101890 0x7f4534101d10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 -- 172.21.15.112:0/1919085838 shutdown_connections 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 --2- 172.21.15.112:0/1919085838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4534102250 0x7f4534108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 --2- 172.21.15.112:0/1919085838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4534101890 0x7f4534101d10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 --2- 172.21.15.112:0/1919085838 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45341006a0 0x7f4534100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 -- 172.21.15.112:0/1919085838 >> 172.21.15.112:0/1919085838 conn(0x7f45340fbc10 msgr2=0x7f45340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 -- 172.21.15.112:0/1919085838 shutdown_connections 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 -- 172.21.15.112:0/1919085838 wait complete. 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 Processor -- start 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 -- start start 2024-04-24T23:14:43.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45341006a0 0x7f4534197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4534101890 0x7f4534198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.114+0000 7f4538abd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4534102250 0x7f453419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f4538abd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f453410d560 con 0x7f45341006a0 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f4538abd640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f453410d3e0 con 0x7f4534101890 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f4538abd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f453410d6e0 con 0x7f4534102250 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45341006a0 0x7f4534197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45341006a0 0x7f4534197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50348/0 (socket says 172.21.15.112:50348) 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 -- 172.21.15.112:0/1353466172 learned_addr learned my addr 172.21.15.112:0/1353466172 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f4531d9b640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4534101890 0x7f4534198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 -- 172.21.15.112:0/1353466172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4534102250 msgr2=0x7f453419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f4532d9d640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4534102250 0x7f453419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4534102250 0x7f453419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 -- 172.21.15.112:0/1353466172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4534101890 msgr2=0x7f4534198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4534101890 0x7f4534198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 -- 172.21.15.112:0/1353466172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453419f840 con 0x7f45341006a0 2024-04-24T23:14:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f4531d9b640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4534101890 0x7f4534198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f453259c640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45341006a0 0x7f4534197c30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f452400d770 tx=0x7f452400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f45237fe640 1 -- 172.21.15.112:0/1353466172 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524017070 con 0x7f45341006a0 2024-04-24T23:14:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f45237fe640 1 -- 172.21.15.112:0/1353466172 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4524009c30 con 0x7f45341006a0 2024-04-24T23:14:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f4538abd640 1 -- 172.21.15.112:0/1353466172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453419fb30 con 0x7f45341006a0 2024-04-24T23:14:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f45237fe640 1 -- 172.21.15.112:0/1353466172 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524003e40 con 0x7f45341006a0 2024-04-24T23:14:43.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.115+0000 7f4538abd640 1 -- 172.21.15.112:0/1353466172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45341a0080 con 0x7f45341006a0 2024-04-24T23:14:43.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.117+0000 7f45237fe640 1 -- 172.21.15.112:0/1353466172 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4524004000 con 0x7f45341006a0 2024-04-24T23:14:43.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453b7d3740 1 Processor -- start 2024-04-24T23:14:43.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453b7d3740 1 -- start start 2024-04-24T23:14:43.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72440 0x55f926c72860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72e30 0x55f926c77930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 0x55f926c7a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453b7d3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f926bd5f00 con 0x55f926c72e30 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453b7d3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f926b7e750 con 0x55f926c77fa0 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453b7d3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f926bd1ec0 con 0x55f926c72440 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72e30 0x55f926c77930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72e30 0x55f926c77930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50364/0 (socket says 172.21.15.112:50364) 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453259c640 1 -- 172.21.15.112:0/3871364925 learned_addr learned my addr 172.21.15.112:0/3871364925 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f45237fe640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f451c0a2b90 0x7f451c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f4531d9b640 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 0x55f926c7a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f4532d9d640 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72440 0x55f926c72860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453259c640 1 -- 172.21.15.112:0/3871364925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72440 msgr2=0x55f926c72860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453259c640 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72440 0x55f926c72860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453259c640 1 -- 172.21.15.112:0/3871364925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 msgr2=0x55f926c7a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453259c640 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 0x55f926c7a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f453259c640 1 -- 172.21.15.112:0/3871364925 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9268ea2d0 con 0x55f926c72e30 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f4532d9d640 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72440 0x55f926c72860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f4532d9d640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f451c0a2b90 0x7f451c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.118+0000 7f45237fe640 1 -- 172.21.15.112:0/1353466172 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f45240ceee0 con 0x7f45341006a0 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.119+0000 7f453259c640 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72e30 0x55f926c77930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4524054df0 tx=0x7f4524022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ef8600c6d0a11d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.119+0000 7f4532d9d640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f451c0a2b90 0x7f451c0a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55f926c73690 tx=0x7f452c009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.119+0000 7f4520ff9640 1 -- 172.21.15.112:0/3871364925 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452409ae90 con 0x55f926c72e30 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.119+0000 7f4520ff9640 1 -- 172.21.15.112:0/3871364925 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4524053410 con 0x55f926c72e30 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.119+0000 7f4520ff9640 1 -- 172.21.15.112:0/3871364925 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524053210 con 0x55f926c72e30 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.119+0000 7f453b7d3740 1 -- 172.21.15.112:0/3871364925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72e30 msgr2=0x55f926c77930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.119+0000 7f453b7d3740 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72e30 0x55f926c77930 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4524054df0 tx=0x7f4524022000 comp rx=0 tx=0).stop 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 -- 172.21.15.112:0/3871364925 shutdown_connections 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 0x55f926c7a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72e30 0x55f926c77930 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 --2- 172.21.15.112:0/3871364925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72440 0x55f926c72860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 -- 172.21.15.112:0/3871364925 >> 172.21.15.112:0/3871364925 conn(0x55f926c6dab0 msgr2=0x55f926c6deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 -- 172.21.15.112:0/3871364925 shutdown_connections 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 -- 172.21.15.112:0/3871364925 wait complete. 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 -- start start 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 Processor -- start 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 0x55f926c7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c7cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c83d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f926c7aef0 con 0x55f926c77fa0 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f926b91cf0 con 0x55f926c72440 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f926c7b070 con 0x55f926c72e30 2024-04-24T23:14:43.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453b7d3740 10 client.0 _ll_register_callbacks cb 0x55f926b916e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f4532d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c83d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f4532d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c83d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50366/0 (socket says 172.21.15.112:50366) 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f4532d9d640 1 -- 172.21.15.112:0/429364427 learned_addr learned my addr 172.21.15.112:0/429364427 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.120+0000 7f453259c640 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 0x55f926c7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4532d9d640 1 -- 172.21.15.112:0/429364427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 msgr2=0x55f926c7cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4531d9b640 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c7cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4532d9d640 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c7cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4532d9d640 1 -- 172.21.15.112:0/429364427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 msgr2=0x55f926c7c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4532d9d640 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 0x55f926c7c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4532d9d640 1 -- 172.21.15.112:0/429364427 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9269b46b0 con 0x55f926c77fa0 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4532d9d640 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c83d60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f452c0096b0 tx=0x7f452c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a75c8746bc28a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:43.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4519ffb640 1 -- 172.21.15.112:0/429364427 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c022070 con 0x55f926c77fa0 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4519ffb640 1 -- 172.21.15.112:0/429364427 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f452c011b60 con 0x55f926c77fa0 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 1 -- 172.21.15.112:0/429364427 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f926be3440 con 0x55f926c77fa0 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4519ffb640 1 -- 172.21.15.112:0/429364427 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f452c02e050 con 0x55f926c77fa0 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 10 client.15495 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 1 -- 172.21.15.112:0/429364427 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f926c86dd0 con 0x55f926c77fa0 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4519ffb640 1 -- 172.21.15.112:0/429364427 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55f926be3440 con 0x55f926c77fa0 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f4519ffb640 1 -- 172.21.15.112:0/429364427 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f452c01a070 con 0x55f926c77fa0 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 10 client.15495 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 20 client.15495 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f926c89550 0x55f926c8a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 4 client.15495 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.121+0000 7f453b7d3740 1 -- 172.21.15.112:0/429364427 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f926bdd790 con 0x55f926c89550 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.122+0000 7f453259c640 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f926c89550 0x55f926c8a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.122+0000 7f453259c640 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f926c89550 0x55f926c8a9c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3eca68e43a300342 server_cookie=1582892003e75af7 in_seq=0 out_seq=0 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.122+0000 7f4519ffb640 10 client.15495 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f4519ffb640 1 -- 172.21.15.112:0/429364427 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f926bdd790 con 0x55f926c89550 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f4519ffb640 10 client.15495 handle_command_reply: tid=0 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 client.15495 shutdown 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 -- 172.21.15.112:0/429364427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 msgr2=0x55f926c83d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c83d60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f452c0096b0 tx=0x7f452c004ab0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 -- 172.21.15.112:0/429364427 shutdown_connections 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f926c89550 0x55f926c8a9c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c83d60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f4519ffb640 0 client.15495 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c7cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 --2- 172.21.15.112:0/429364427 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 0x55f926c7c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 -- 172.21.15.112:0/429364427 >> 172.21.15.112:0/429364427 conn(0x55f926c6dab0 msgr2=0x55f926c6cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 -- 172.21.15.112:0/429364427 shutdown_connections 2024-04-24T23:14:43.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 1 -- 172.21.15.112:0/429364427 wait complete. 2024-04-24T23:14:43.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.123+0000 7f453b7d3740 20 client.15495 trim_cache size 0 max 16384 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f453b7d3740 1 Processor -- start 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f453b7d3740 1 -- start start 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 0x55f926c68690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c83c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c7c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f453b7d3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f926c88d00 con 0x55f926c77fa0 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f453b7d3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f926c893d0 con 0x55f926c72440 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f453b7d3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f926c89aa0 con 0x55f926c72e30 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c7c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c7c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50378/0 (socket says 172.21.15.112:50378) 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 -- 172.21.15.112:0/3058863287 learned_addr learned my addr 172.21.15.112:0/3058863287 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 -- 172.21.15.112:0/3058863287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 msgr2=0x55f926c83c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 --2- 172.21.15.112:0/3058863287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c83c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 -- 172.21.15.112:0/3058863287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 msgr2=0x55f926c68690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 --2- 172.21.15.112:0/3058863287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 0x55f926c68690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 -- 172.21.15.112:0/3058863287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f926c736c0 con 0x55f926c77fa0 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f4532d9d640 1 --2- 172.21.15.112:0/3058863287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c7c7b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f452c006010 tx=0x7f452c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1101bbe4ce14d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.141+0000 7f451bfff640 1 -- 172.21.15.112:0/3058863287 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c022070 con 0x55f926c77fa0 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f451bfff640 1 -- 172.21.15.112:0/3058863287 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f452c0266a0 con 0x55f926c77fa0 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f451bfff640 1 -- 172.21.15.112:0/3058863287 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c016070 con 0x55f926c77fa0 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 -- 172.21.15.112:0/3058863287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 msgr2=0x55f926c7c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 --2- 172.21.15.112:0/3058863287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c7c7b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f452c006010 tx=0x7f452c009cd0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 -- 172.21.15.112:0/3058863287 shutdown_connections 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 --2- 172.21.15.112:0/3058863287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c77fa0 0x55f926c7c7b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 --2- 172.21.15.112:0/3058863287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c83c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 --2- 172.21.15.112:0/3058863287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c72440 0x55f926c68690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 -- 172.21.15.112:0/3058863287 >> 172.21.15.112:0/3058863287 conn(0x55f926c6dab0 msgr2=0x55f926c6de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 -- 172.21.15.112:0/3058863287 shutdown_connections 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 -- 172.21.15.112:0/3058863287 wait complete. 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 -- start start 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.142+0000 7f453b7d3740 1 Processor -- start 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72440 0x55f926c763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c76910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 0x55f926c82400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f926c8a840 con 0x55f926c72440 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f926c8a170 con 0x55f926c77fa0 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f926c8af10 con 0x55f926c72e30 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 10 client.0 _ll_register_callbacks cb 0x7f452c009f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72440 0x55f926c763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72440 0x55f926c763d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50380/0 (socket says 172.21.15.112:50380) 2024-04-24T23:14:43.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 -- 172.21.15.112:0/1662924199 learned_addr learned my addr 172.21.15.112:0/1662924199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f4531d9b640 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c76910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 -- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 msgr2=0x55f926c76910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c76910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 -- 172.21.15.112:0/1662924199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 msgr2=0x55f926c82400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 0x55f926c82400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 -- 172.21.15.112:0/1662924199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f926c829f0 con 0x55f926c72440 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f4531d9b640 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c76910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453259c640 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72440 0x55f926c763d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f45240cc960 tx=0x7f45240036e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36ffeafcdb6c630 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f4519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f4519ffb640 1 -- 172.21.15.112:0/1662924199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524020040 con 0x55f926c72440 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f4519ffb640 1 -- 172.21.15.112:0/1662924199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f45240af570 con 0x55f926c72440 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 1 -- 172.21.15.112:0/1662924199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f452c0266a0 con 0x55f926c72440 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f4519ffb640 1 -- 172.21.15.112:0/1662924199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4524007600 con 0x55f926c72440 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 10 client.15501 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:43.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.143+0000 7f453b7d3740 1 -- 172.21.15.112:0/1662924199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f926c83cf0 con 0x55f926c72440 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f4519ffb640 1 -- 172.21.15.112:0/1662924199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f452c0266a0 con 0x55f926c72440 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f4519ffb640 1 -- 172.21.15.112:0/1662924199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4524097930 con 0x55f926c72440 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f453b7d3740 10 client.15501 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f453b7d3740 10 client.15501 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f453b7d3740 20 client.15501 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f453b7d3740 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f926cf3430 0x55f926cf2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f453b7d3740 4 client.15501 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f453b7d3740 1 -- 172.21.15.112:0/1662924199 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "status"}) -- 0x55f926b5bb80 con 0x55f926cf3430 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f4531d9b640 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f926cf3430 0x55f926cf2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f4531d9b640 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f926cf3430 0x55f926cf2140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84b0c819d1813785 server_cookie=28fe11a7dc7abfe9 in_seq=0 out_seq=0 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.144+0000 7f4519ffb640 10 client.15501 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4519ffb640 1 -- 172.21.15.112:0/1662924199 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f926b5bb80 con 0x55f926cf3430 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4519ffb640 10 client.15501 handle_command_reply: tid=0 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 client.15501 shutdown 2024-04-24T23:14:43.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 -- 172.21.15.112:0/1662924199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72440 msgr2=0x55f926c763d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72440 0x55f926c763d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f45240cc960 tx=0x7f45240036e0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 -- 172.21.15.112:0/1662924199 shutdown_connections 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55f926cf3430 0x55f926cf2140 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f926c77fa0 0x55f926c82400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4519ffb640 0 client.15501 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f926c72e30 0x55f926c76910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 --2- 172.21.15.112:0/1662924199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f926c72440 0x55f926c763d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 -- 172.21.15.112:0/1662924199 >> 172.21.15.112:0/1662924199 conn(0x55f926c6dab0 msgr2=0x55f926c68690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 -- 172.21.15.112:0/1662924199 shutdown_connections 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 1 -- 172.21.15.112:0/1662924199 wait complete. 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f453b7d3740 20 client.15501 trim_cache size 0 max 16384 2024-04-24T23:14:43.153 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24490, 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 111.266803581, 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 150.35998698399999 2024-04-24T23:14:43.154 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:43.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 -- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f451c0a2b90 msgr2=0x7f451c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f451c0a2b90 0x7f451c0a5050 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55f926c73690 tx=0x7f452c009510 comp rx=0 tx=0).stop 2024-04-24T23:14:43.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 -- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45341006a0 msgr2=0x7f4534197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45341006a0 0x7f4534197c30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f452400d770 tx=0x7f452400dc30 comp rx=0 tx=0).stop 2024-04-24T23:14:43.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 -- 172.21.15.112:0/1353466172 shutdown_connections 2024-04-24T23:14:43.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f451c0a2b90 0x7f451c0a5050 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4534102250 0x7f453419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4534101890 0x7f4534198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 --2- 172.21.15.112:0/1353466172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45341006a0 0x7f4534197c30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 -- 172.21.15.112:0/1353466172 >> 172.21.15.112:0/1353466172 conn(0x7f45340fbc10 msgr2=0x7f4534104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 -- 172.21.15.112:0/1353466172 shutdown_connections 2024-04-24T23:14:43.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.145+0000 7f4538abd640 1 -- 172.21.15.112:0/1353466172 wait complete. 2024-04-24T23:14:43.157 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:43 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/802791993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:43.157 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:43 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2110042075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:43.158 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-24T23:14:43.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 -- 172.21.15.112:0/3652082093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b01006a0 msgr2=0x7f93b0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 --2- 172.21.15.112:0/3652082093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b01006a0 0x7f93b0100ac0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009900 tx=0x7f93a002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 -- 172.21.15.112:0/3652082093 shutdown_connections 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 --2- 172.21.15.112:0/3652082093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b0102250 0x7f93b0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 --2- 172.21.15.112:0/3652082093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b0101890 0x7f93b0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 --2- 172.21.15.112:0/3652082093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b01006a0 0x7f93b0100ac0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 -- 172.21.15.112:0/3652082093 >> 172.21.15.112:0/3652082093 conn(0x7f93b00fbc30 msgr2=0x7f93b00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 -- 172.21.15.112:0/3652082093 shutdown_connections 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.255+0000 7f93b699e640 1 -- 172.21.15.112:0/3652082093 wait complete. 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93b699e640 1 Processor -- start 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93b699e640 1 -- start start 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93b699e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b01006a0 0x7f93b0104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93b699e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b0101890 0x7f93b0102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93b699e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b0102250 0x7f93b01031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93b699e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010c4a0 con 0x7f93b0101890 2024-04-24T23:14:43.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93b699e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010c320 con 0x7f93b01006a0 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93b699e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010c620 con 0x7f93b0102250 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b0101890 0x7f93b0102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b0101890 0x7f93b0102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50398/0 (socket says 172.21.15.112:50398) 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93af7fe640 1 -- 172.21.15.112:0/2785630624 learned_addr learned my addr 172.21.15.112:0/2785630624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93af7fe640 1 -- 172.21.15.112:0/2785630624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b0102250 msgr2=0x7f93b01031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93af7fe640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b0102250 0x7f93b01031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93af7fe640 1 -- 172.21.15.112:0/2785630624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b01006a0 msgr2=0x7f93b0104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.256+0000 7f93af7fe640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b01006a0 0x7f93b0104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.257+0000 7f93af7fe640 1 -- 172.21.15.112:0/2785630624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b0103a40 con 0x7f93b0101890 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.257+0000 7f93af7fe640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b0101890 0x7f93b0102c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f93a400d950 tx=0x7f93a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.257+0000 7f93ad7fa640 1 -- 172.21.15.112:0/2785630624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4017070 con 0x7f93b0101890 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.257+0000 7f93ad7fa640 1 -- 172.21.15.112:0/2785630624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f93a4009c30 con 0x7f93b0101890 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.257+0000 7f93b699e640 1 -- 172.21.15.112:0/2785630624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b01a1dc0 con 0x7f93b0101890 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.257+0000 7f93ad7fa640 1 -- 172.21.15.112:0/2785630624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a400f740 con 0x7f93b0101890 2024-04-24T23:14:43.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.257+0000 7f93b699e640 1 -- 172.21.15.112:0/2785630624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b01a2280 con 0x7f93b0101890 2024-04-24T23:14:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93ad7fa640 1 -- 172.21.15.112:0/2785630624 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f93a4010a80 con 0x7f93b0101890 2024-04-24T23:14:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93b9789740 1 Processor -- start 2024-04-24T23:14:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93b9789740 1 -- start start 2024-04-24T23:14:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93b9789740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4440 0x55c89d3b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93b9789740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3b9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93b9789740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93b9789740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c89d317f00 con 0x55c89d3b9fa0 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93b9789740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c89d2c0750 con 0x55c89d3b4440 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93b9789740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c89d313ec0 con 0x55c89d3b4e30 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.259+0000 7f93ad7fa640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93980a2c10 0x7f93980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3bc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50408/0 (socket says 172.21.15.112:50408) 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 -- 172.21.15.112:0/3198537063 learned_addr learned my addr 172.21.15.112:0/3198537063 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93b4f3b640 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4440 0x55c89d3b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93b4f3b640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93980a2c10 0x7f93980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93affff640 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93ad7fa640 1 -- 172.21.15.112:0/2785630624 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f93a40cf0f0 con 0x7f93b0101890 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 -- 172.21.15.112:0/3198537063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 msgr2=0x55c89d3b9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3b9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 -- 172.21.15.112:0/3198537063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4440 msgr2=0x55c89d3b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4440 0x55c89d3b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 -- 172.21.15.112:0/3198537063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89d02c2d0 con 0x55c89d3b9fa0 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93affff640 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93af7fe640 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3bc430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f93a40cc800 tx=0x7f93a4010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f19df26baabd7d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93b4f3b640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93980a2c10 0x7f93980a50d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55c89d3b5690 tx=0x7f93a8005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93967fc640 1 -- 172.21.15.112:0/3198537063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4020070 con 0x55c89d3b9fa0 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93967fc640 1 -- 172.21.15.112:0/3198537063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f93a409b560 con 0x55c89d3b9fa0 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.260+0000 7f93967fc640 1 -- 172.21.15.112:0/3198537063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a409ac10 con 0x55c89d3b9fa0 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 -- 172.21.15.112:0/3198537063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 msgr2=0x55c89d3bc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3bc430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f93a40cc800 tx=0x7f93a4010e20 comp rx=0 tx=0).stop 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 -- 172.21.15.112:0/3198537063 shutdown_connections 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3bc430 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 --2- 172.21.15.112:0/3198537063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4440 0x55c89d3b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 -- 172.21.15.112:0/3198537063 >> 172.21.15.112:0/3198537063 conn(0x55c89d3afab0 msgr2=0x55c89d3afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 -- 172.21.15.112:0/3198537063 shutdown_connections 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 -- 172.21.15.112:0/3198537063 wait complete. 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 -- start start 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.261+0000 7f93b9789740 1 Processor -- start 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b9789740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b9789740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 0x55c89d3becb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b9789740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b9fa0 0x55c89d3c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b9789740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c89d3bce90 con 0x55c89d3b4440 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b9789740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c89d2d3cf0 con 0x55c89d3b4e30 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b9789740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c89d3bd010 con 0x55c89d3b9fa0 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b9789740 10 client.0 _ll_register_callbacks cb 0x55c89d2d36e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3be770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50422/0 (socket says 172.21.15.112:50422) 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 -- 172.21.15.112:0/465234611 learned_addr learned my addr 172.21.15.112:0/465234611 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b4f3b640 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b9fa0 0x55c89d3c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 -- 172.21.15.112:0/465234611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b9fa0 msgr2=0x55c89d3c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b9fa0 0x55c89d3c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 -- 172.21.15.112:0/465234611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 msgr2=0x55c89d3becb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93af7fe640 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 0x55c89d3becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 0x55c89d3becb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 -- 172.21.15.112:0/465234611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89d0f66b0 con 0x55c89d3b4440 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93affff640 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3be770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009900 tx=0x7f93a0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da9f8194d188d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f938f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f938f7fe640 1 -- 172.21.15.112:0/465234611 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0040070 con 0x55c89d3b4440 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f938f7fe640 1 -- 172.21.15.112:0/465234611 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f93a0031940 con 0x55c89d3b4440 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.262+0000 7f93b9789740 1 -- 172.21.15.112:0/465234611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a409b560 con 0x55c89d3b4440 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f938f7fe640 1 -- 172.21.15.112:0/465234611 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93a004c050 con 0x55c89d3b4440 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93b9789740 10 client.15513 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93b9789740 1 -- 172.21.15.112:0/465234611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c89d3c8d40 con 0x55c89d3b4440 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f938f7fe640 1 -- 172.21.15.112:0/465234611 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f93a409b560 con 0x55c89d3b4440 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f938f7fe640 1 -- 172.21.15.112:0/465234611 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93a003a040 con 0x55c89d3b4440 2024-04-24T23:14:43.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93b9789740 10 client.15513 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93b9789740 10 client.15513 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93b9789740 20 client.15513 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93b9789740 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c89d3cb3f0 0x55c89d3cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93b9789740 4 client.15513 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93b9789740 1 -- 172.21.15.112:0/465234611 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c89d325440 con 0x55c89d3cb3f0 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.263+0000 7f93af7fe640 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c89d3cb3f0 0x55c89d3cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f93af7fe640 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c89d3cb3f0 0x55c89d3cc860 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb993ace83666efe server_cookie=7ccefef7004fe0aa in_seq=0 out_seq=0 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f938f7fe640 10 client.15513 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f938f7fe640 1 -- 172.21.15.112:0/465234611 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c89d325440 con 0x55c89d3cb3f0 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f938f7fe640 10 client.15513 handle_command_reply: tid=0 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f93b9789740 1 client.15513 shutdown 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f93b9789740 1 -- 172.21.15.112:0/465234611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 msgr2=0x55c89d3be770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f93b9789740 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3be770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009900 tx=0x7f93a0005270 comp rx=0 tx=0).stop 2024-04-24T23:14:43.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f93b9789740 1 -- 172.21.15.112:0/465234611 shutdown_connections 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.264+0000 7f93b9789740 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c89d3cb3f0 0x55c89d3cc860 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.265+0000 7f93b9789740 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b9fa0 0x55c89d3c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.265+0000 7f93b9789740 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 0x55c89d3becb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.265+0000 7f938f7fe640 0 client.15513 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.265+0000 7f93b9789740 1 --2- 172.21.15.112:0/465234611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3be770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.265+0000 7f93b9789740 1 -- 172.21.15.112:0/465234611 >> 172.21.15.112:0/465234611 conn(0x55c89d3afab0 msgr2=0x55c89d3aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.265+0000 7f93b9789740 1 -- 172.21.15.112:0/465234611 shutdown_connections 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.265+0000 7f93b9789740 1 -- 172.21.15.112:0/465234611 wait complete. 2024-04-24T23:14:43.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.265+0000 7f93b9789740 20 client.15513 trim_cache size 0 max 16384 2024-04-24T23:14:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b9789740 1 Processor -- start 2024-04-24T23:14:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b9789740 1 -- start start 2024-04-24T23:14:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b9789740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4440 0x55c89d3aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b9789740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 0x55c89d3c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b9789740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b9789740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c89d3caba0 con 0x55c89d3b9fa0 2024-04-24T23:14:43.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b9789740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c89d3cb270 con 0x55c89d3b4e30 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b9789740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c89d3cb940 con 0x55c89d3b4440 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3ae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50426/0 (socket says 172.21.15.112:50426) 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 -- 172.21.15.112:0/3499808446 learned_addr learned my addr 172.21.15.112:0/3499808446 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93affff640 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4440 0x55c89d3aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 -- 172.21.15.112:0/3499808446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4440 msgr2=0x55c89d3aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4440 0x55c89d3aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 -- 172.21.15.112:0/3499808446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 msgr2=0x55c89d3c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93af7fe640 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 0x55c89d3c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 0x55c89d3c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 -- 172.21.15.112:0/3499808446 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c89d3b9cf0 con 0x55c89d3b9fa0 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f93b4f3b640 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3ae250 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f93a8011ae0 tx=0x7f93a80094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e0db0b371a0439 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.283+0000 7f9395ffb640 1 -- 172.21.15.112:0/3499808446 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8025070 con 0x55c89d3b9fa0 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f9395ffb640 1 -- 172.21.15.112:0/3499808446 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f93a801ee50 con 0x55c89d3b9fa0 2024-04-24T23:14:43.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f9395ffb640 1 -- 172.21.15.112:0/3499808446 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a80046d0 con 0x55c89d3b9fa0 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 -- 172.21.15.112:0/3499808446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 msgr2=0x55c89d3ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3ae250 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f93a8011ae0 tx=0x7f93a80094a0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 -- 172.21.15.112:0/3499808446 shutdown_connections 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b9fa0 0x55c89d3ae250 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b4e30 0x55c89d3c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 --2- 172.21.15.112:0/3499808446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4440 0x55c89d3aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 -- 172.21.15.112:0/3499808446 >> 172.21.15.112:0/3499808446 conn(0x55c89d3afab0 msgr2=0x55c89d3afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 -- 172.21.15.112:0/3499808446 shutdown_connections 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 -- 172.21.15.112:0/3499808446 wait complete. 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.284+0000 7f93b9789740 1 -- start start 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93b9789740 1 Processor -- start 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93b9789740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93b9789740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3c2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93b9789740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b9fa0 0x55c89d3c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93b9789740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c89d3cc6e0 con 0x55c89d3b4440 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93b9789740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c89d3cc010 con 0x55c89d3b9fa0 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93b9789740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c89d3ccdb0 con 0x55c89d3b4e30 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93b9789740 10 client.0 _ll_register_callbacks cb 0x7f93a0004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50438/0 (socket says 172.21.15.112:50438) 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 -- 172.21.15.112:0/249383145 learned_addr learned my addr 172.21.15.112:0/249383145 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93af7fe640 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 -- 172.21.15.112:0/249383145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 msgr2=0x55c89d3c2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3c2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 -- 172.21.15.112:0/249383145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b9fa0 msgr2=0x55c89d3c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b9fa0 0x55c89d3c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 -- 172.21.15.112:0/249383145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c89d3c5440 con 0x55c89d3b4440 2024-04-24T23:14:43.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f93affff640 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3c1ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f93a0031d00 tx=0x7f93a0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fa0517dee221c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f938f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.285+0000 7f938f7fe640 1 -- 172.21.15.112:0/249383145 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0040040 con 0x55c89d3b4440 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 1 -- 172.21.15.112:0/249383145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a801ee50 con 0x55c89d3b4440 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f938f7fe640 1 -- 172.21.15.112:0/249383145 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f93a0003990 con 0x55c89d3b4440 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f938f7fe640 1 -- 172.21.15.112:0/249383145 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93a0060050 con 0x55c89d3b4440 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 10 client.15519 fetch_fsmap learned FSMap version 27 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 1 -- 172.21.15.112:0/249383145 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c89d4318c0 con 0x55c89d3b4440 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f938f7fe640 1 -- 172.21.15.112:0/249383145 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f93a0054020 con 0x55c89d3b4440 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f938f7fe640 1 -- 172.21.15.112:0/249383145 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93a0003b90 con 0x55c89d3b4440 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 10 client.15519 fetch_fsmap finished waiting for FSMap version 27 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 10 client.15519 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 20 client.15519 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c89d436540 0x55c89d435250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:43.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 4 client.15519 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93b9789740 1 -- 172.21.15.112:0/249383145 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "status"}) -- 0x55c89d3bd1a0 con 0x55c89d436540 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.286+0000 7f93af7fe640 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c89d436540 0x55c89d435250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f938f7fe640 10 client.15519 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93af7fe640 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c89d436540 0x55c89d435250 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67fa4d08a3c750db server_cookie=d16e9b2d23644952 in_seq=0 out_seq=0 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f938f7fe640 1 -- 172.21.15.112:0/249383145 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c89d3bd1a0 con 0x55c89d436540 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f938f7fe640 10 client.15519 handle_command_reply: tid=0 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 client.15519 shutdown 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 -- 172.21.15.112:0/249383145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 msgr2=0x55c89d3c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3c1ee0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f93a0031d00 tx=0x7f93a0003820 comp rx=0 tx=0).stop 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 -- 172.21.15.112:0/249383145 shutdown_connections 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c89d436540 0x55c89d435250 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c89d3b9fa0 0x55c89d3c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f938f7fe640 0 client.15519 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c89d3b4e30 0x55c89d3c2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 --2- 172.21.15.112:0/249383145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c89d3b4440 0x55c89d3c1ee0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 -- 172.21.15.112:0/249383145 >> 172.21.15.112:0/249383145 conn(0x55c89d3afab0 msgr2=0x55c89d3aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 -- 172.21.15.112:0/249383145 shutdown_connections 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 1 -- 172.21.15.112:0/249383145 wait complete. 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.287+0000 7f93b9789740 20 client.15519 trim_cache size 0 max 16384 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "e62ba378-028e-11ef-bc93-c7b262605968", 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24524, 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-24T23:14:43.296 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-24T23:14:43.297 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 135.30376261699999, 2024-04-24T23:14:43.297 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 27, 2024-04-24T23:14:43.297 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 94, 2024-04-24T23:14:43.297 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 90, 2024-04-24T23:14:43.297 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 141.850334358 2024-04-24T23:14:43.297 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 -- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93980a2c10 msgr2=0x7f93980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93980a2c10 0x7f93980a50d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55c89d3b5690 tx=0x7f93a8005e00 comp rx=0 tx=0).stop 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 -- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b0101890 msgr2=0x7f93b0102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b0101890 0x7f93b0102c70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f93a400d950 tx=0x7f93a400de10 comp rx=0 tx=0).stop 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 -- 172.21.15.112:0/2785630624 shutdown_connections 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f93980a2c10 0x7f93980a50d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93b0102250 0x7f93b01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f93b0101890 0x7f93b0102c70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 --2- 172.21.15.112:0/2785630624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93b01006a0 0x7f93b0104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 -- 172.21.15.112:0/2785630624 >> 172.21.15.112:0/2785630624 conn(0x7f93b00fbc30 msgr2=0x7f93b00fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 -- 172.21.15.112:0/2785630624 shutdown_connections 2024-04-24T23:14:43.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:43.288+0000 7f93b699e640 1 -- 172.21.15.112:0/2785630624 wait complete. 2024-04-24T23:14:43.301 INFO:teuthology.run_tasks:Running task sequential... 2024-04-24T23:14:43.311 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-24T23:14:43.311 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-24T23:14:43.312 INFO:teuthology.task.print:Enabling mgr modules 2024-04-24T23:14:43.312 INFO:teuthology.run_tasks:Running task kclient... 2024-04-24T23:14:43.332 INFO:tasks.kclient:Mounting kernel clients... 2024-04-24T23:14:43.333 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-24T23:14:43.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:43.333 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-24T23:14:43.333 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:43 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/802791993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:43.334 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:43 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2110042075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:43.348 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-24T23:14:43.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:43.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-24T23:14:43.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:43.412 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-24T23:14:43.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:43 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/802791993' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:43.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:43 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2110042075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:43.501 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'rbytes': False, 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-24T23:14:43.501 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:14:43.501 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:14:43.502 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:14:43.741 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:14:44.140 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:44 smithi112 ceph-mon[29186]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:44 smithi150 ceph-mon[32247]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:44 smithi187 ceph-mon[33292]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:14:45.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.448+0000 7f83fa584700 1 -- 172.21.15.112:0/1430087837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4101ce0 msgr2=0x7f83f4102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.448+0000 7f83fa584700 1 --2- 172.21.15.112:0/1430087837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4101ce0 0x7f83f4102160 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f83e80099c0 tx=0x7f83e8009cd0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.448+0000 7f83fa584700 1 -- 172.21.15.112:0/1430087837 shutdown_connections 2024-04-24T23:14:45.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.448+0000 7f83fa584700 1 --2- 172.21.15.112:0/1430087837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4107400 0x7f83f41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.448+0000 7f83fa584700 1 --2- 172.21.15.112:0/1430087837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f41026a0 0x7f83f410e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.448+0000 7f83fa584700 1 --2- 172.21.15.112:0/1430087837 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f4101ce0 0x7f83f4102160 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.448+0000 7f83fa584700 1 -- 172.21.15.112:0/1430087837 >> 172.21.15.112:0/1430087837 conn(0x7f83f4076450 msgr2=0x7f83f4076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:45.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.449+0000 7f83fa584700 1 -- 172.21.15.112:0/1430087837 shutdown_connections 2024-04-24T23:14:45.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.449+0000 7f83fa584700 1 -- 172.21.15.112:0/1430087837 wait complete. 2024-04-24T23:14:45.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.449+0000 7f83fa584700 1 Processor -- start 2024-04-24T23:14:45.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.449+0000 7f83fa584700 1 -- start start 2024-04-24T23:14:45.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83fa584700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f4101ce0 0x7f83f419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83fa584700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f41026a0 0x7f83f419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83fa584700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4107400 0x7f83f4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83fa584700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83f41117b0 con 0x7f83f41026a0 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83fa584700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4111630 con 0x7f83f4107400 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83fa584700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4111930 con 0x7f83f4101ce0 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f41026a0 0x7f83f419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f41026a0 0x7f83f419ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50464/0 (socket says 172.21.15.112:50464) 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 -- 172.21.15.112:0/2861248192 learned_addr learned my addr 172.21.15.112:0/2861248192 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f3fff700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f4101ce0 0x7f83f419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 -- 172.21.15.112:0/2861248192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f4101ce0 msgr2=0x7f83f419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f4101ce0 0x7f83f419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 -- 172.21.15.112:0/2861248192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4107400 msgr2=0x7f83f4198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f8b21700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4107400 0x7f83f4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4107400 0x7f83f4198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 -- 172.21.15.112:0/2861248192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f41991e0 con 0x7f83f41026a0 2024-04-24T23:14:45.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f8b21700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4107400 0x7f83f4198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:45.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f37fe700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f41026a0 0x7f83f419ee00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f83e800b580 tx=0x7f83e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:45.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f17fa700 1 -- 172.21.15.112:0/2861248192 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8027070 con 0x7f83f41026a0 2024-04-24T23:14:45.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f17fa700 1 -- 172.21.15.112:0/2861248192 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f83e80041c0 con 0x7f83f41026a0 2024-04-24T23:14:45.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f4199470 con 0x7f83f41026a0 2024-04-24T23:14:45.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.450+0000 7f83f17fa700 1 -- 172.21.15.112:0/2861248192 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e800f6c0 con 0x7f83f41026a0 2024-04-24T23:14:45.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.451+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f41a5790 con 0x7f83f41026a0 2024-04-24T23:14:45.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.451+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f4068f20 con 0x7f83f41026a0 2024-04-24T23:14:45.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.453+0000 7f83f17fa700 1 -- 172.21.15.112:0/2861248192 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f83e802a070 con 0x7f83f41026a0 2024-04-24T23:14:45.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.454+0000 7f83f17fa700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f83dc0a2ca0 0x7f83dc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.454+0000 7f83f17fa700 1 -- 172.21.15.112:0/2861248192 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f83e80d08f0 con 0x7f83f41026a0 2024-04-24T23:14:45.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.454+0000 7f83f3fff700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f83dc0a2ca0 0x7f83dc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.454+0000 7f83f3fff700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f83dc0a2ca0 0x7f83dc0a5160 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83f4199bc0 tx=0x7f83e4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:45.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.458+0000 7f83f17fa700 1 -- 172.21.15.112:0/2861248192 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f83e8031070 con 0x7f83f41026a0 2024-04-24T23:14:45.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.698+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f83f41a5960 con 0x7f83f41026a0 2024-04-24T23:14:45.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.698+0000 7f83f17fa700 1 -- 172.21.15.112:0/2861248192 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f83e8013070 con 0x7f83f41026a0 2024-04-24T23:14:45.701 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.700+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f83dc0a2ca0 msgr2=0x7f83dc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.700+0000 7f83fa584700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f83dc0a2ca0 0x7f83dc0a5160 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83f4199bc0 tx=0x7f83e4009190 comp rx=0 tx=0).stop 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.700+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f41026a0 msgr2=0x7f83f419ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.700+0000 7f83fa584700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f41026a0 0x7f83f419ee00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f83e800b580 tx=0x7f83e80047b0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.701+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 shutdown_connections 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.701+0000 7f83fa584700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f83dc0a2ca0 0x7f83dc0a5160 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.701+0000 7f83fa584700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f83f4107400 0x7f83f4198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.701+0000 7f83fa584700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f4101ce0 0x7f83f419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.701+0000 7f83fa584700 1 --2- 172.21.15.112:0/2861248192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f41026a0 0x7f83f419ee00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.701+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 >> 172.21.15.112:0/2861248192 conn(0x7f83f4076450 msgr2=0x7f83f41054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.701+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 shutdown_connections 2024-04-24T23:14:45.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:45.701+0000 7f83fa584700 1 -- 172.21.15.112:0/2861248192 wait complete. 2024-04-24T23:14:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:46 smithi112 ceph-mon[29186]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:46 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2861248192' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:14:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:46 smithi150 ceph-mon[32247]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:46 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2861248192' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:14:46.390 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:14:46.390 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:14:46.390 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-24T23:14:46.390 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:14:46.390 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:14:46.390 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:14:46.390 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:14:46.391 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:14:46.391 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:14:46.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:46.391 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-24T23:14:46.406 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:14:46.406 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:14:46.406 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:14:46.407 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-24T23:14:46.408 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42105sec preferred_lft 42105sec 2024-04-24T23:14:46.408 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5787:d070:61ba:675f/64 scope link noprefixroute 2024-04-24T23:14:46.408 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:14:46.408 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:14:46.408 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:14:46.408 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:14:46.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:14:46.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-04-24T23:14:46.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-04-24T23:14:46.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:14:46.408 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:14:46.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:46 smithi187 ceph-mon[33292]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:14:46.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:46 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2861248192' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:14:46.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:14:46.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:46.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:46.544 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:14:46.566 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-24T23:14:46.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:46.568 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-24T23:14:46.624 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-24T23:14:46.625 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:14:46.625 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-24T23:14:46.625 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-24T23:14:46.625 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:14:46.625 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:14:46.625 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:14:46.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-24T23:14:46.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-24T23:14:46.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-24T23:14:46.626 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:14:46.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:14:46.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:46.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:46.748 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-24T23:14:46.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:46.763 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-04-24T23:14:46.817 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:14:46.817 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:14:46.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:14:46.818 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T23:14:46.818 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:14:46.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:14:46.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:46.933 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T23:14:46.933 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:14:46.933 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:14:46.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T23:14:46.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:14:46.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T23:14:46.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T23:14:46.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T23:14:46.934 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:14:46.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:14:47.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:47.043 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:14:47.043 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-24T23:14:47.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-04-24T23:14:47.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:14:47.043 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-24T23:14:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:14:47.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:47.118 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:14:47.119 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:14:47.133 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:14:47.133 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:14:47.133 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:14:47.187 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-24T23:14:47.187 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-24T23:14:47.187 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/dfd928e6-5bb8-4127-82af-d599ace70756 2024-04-24T23:14:47.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:47.187 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/dfd928e6-5bb8-4127-82af-d599ace70756 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-24T23:14:47.268 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-24T23:14:47.268 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-24T23:14:47.391 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2024-04-24T23:14:47.391 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2024-04-24T23:14:47.391 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.150:3300,172.21.15.187:3300" 2024-04-24T23:14:47.391 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 0@e62ba378-028e-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/dfd928e6-5bb8-4127-82af-d599ace70756 2024-04-24T23:14:47.391 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.112:3300/172.21.15.150:3300/172.21.15.187:3300" will pass to kernel 2024-04-24T23:14:47.392 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-24T23:14:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-04-24T23:14:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-24T23:14:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-04-24T23:14:47.392 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-24T23:14:47.393 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-24T23:14:47.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:47.394 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:14:47.460 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:14:47.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:14:47.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:47.557 INFO:teuthology.orchestra.run.smithi112.stdout:client24994 2024-04-24T23:14:47.557 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:14:47.557 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:14:47.574 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:14:47.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:14:47.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:47.669 INFO:teuthology.orchestra.run.smithi112.stdout:client24994 2024-04-24T23:14:47.670 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-24T23:14:47.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-24T23:14:47.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:47.725 INFO:teuthology.orchestra.run.smithi112.stdout:client24994 2024-04-24T23:14:47.725 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e62ba378-028e-11ef-bc93-c7b262605968.client24994/status') 2024-04-24T23:14:47.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:14:47.782 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.24994 (3)172.21.15.112:0/3522259867 2024-04-24T23:14:47.783 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-04-24T23:14:47.783 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-04-24T23:14:47.783 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-04-24T23:14:47.783 INFO:teuthology.orchestra.run.smithi112.stderr:72 bytes copied, 3.8693e-05 s, 1.9 MB/s 2024-04-24T23:14:47.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:14:47.790 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T23:14:47.800 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T23:14:47.800 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-24T23:14:47.801 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-24T23:14:47.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 -- 172.21.15.112:0/3139742681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc1022e0 msgr2=0x7fabdc1046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3139742681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc1022e0 0x7fabdc1046d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009900 tx=0x7fabd402a3f0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 -- 172.21.15.112:0/3139742681 shutdown_connections 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3139742681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabdc107540 0x7fabdc109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3139742681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabdc104c10 0x7fabdc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3139742681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc1022e0 0x7fabdc1046d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 -- 172.21.15.112:0/3139742681 >> 172.21.15.112:0/3139742681 conn(0x7fabdc0fbc10 msgr2=0x7fabdc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 -- 172.21.15.112:0/3139742681 shutdown_connections 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.898+0000 7fabe3f63640 1 -- 172.21.15.112:0/3139742681 wait complete. 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe3f63640 1 Processor -- start 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe3f63640 1 -- start start 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe3f63640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabdc1022e0 0x7fabdc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe3f63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc104c10 0x7fabdc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe3f63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabdc107540 0x7fabdc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:47.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe3f63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc10d560 con 0x7fabdc104c10 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe3f63640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc10d3e0 con 0x7fabdc1022e0 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe3f63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc10d6e0 con 0x7fabdc107540 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc104c10 0x7fabdc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc104c10 0x7fabdc197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50488/0 (socket says 172.21.15.112:50488) 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 -- 172.21.15.112:0/3811448228 learned_addr learned my addr 172.21.15.112:0/3811448228 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 -- 172.21.15.112:0/3811448228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabdc107540 msgr2=0x7fabdc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe2500640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabdc107540 0x7fabdc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabdc107540 0x7fabdc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 -- 172.21.15.112:0/3811448228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabdc1022e0 msgr2=0x7fabdc197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabdc1022e0 0x7fabdc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 -- 172.21.15.112:0/3811448228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc19f680 con 0x7fabdc104c10 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe2500640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabdc107540 0x7fabdc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.899+0000 7fabe14fe640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc104c10 0x7fabdc197fa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00e9c0 tx=0x7fabcc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:47.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.900+0000 7fabd2ffd640 1 -- 172.21.15.112:0/3811448228 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00cd80 con 0x7fabdc104c10 2024-04-24T23:14:47.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.900+0000 7fabd2ffd640 1 -- 172.21.15.112:0/3811448228 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fabcc005520 con 0x7fabdc104c10 2024-04-24T23:14:47.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.900+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc19f970 con 0x7fabdc104c10 2024-04-24T23:14:47.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.900+0000 7fabd2ffd640 1 -- 172.21.15.112:0/3811448228 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc010690 con 0x7fabdc104c10 2024-04-24T23:14:47.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.900+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc19fec0 con 0x7fabdc104c10 2024-04-24T23:14:47.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.901+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc0fd700 con 0x7fabdc104c10 2024-04-24T23:14:47.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.902+0000 7fabd2ffd640 1 -- 172.21.15.112:0/3811448228 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fabcc0046c0 con 0x7fabdc104c10 2024-04-24T23:14:47.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.902+0000 7fabd2ffd640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabc80a2c30 0x7fabc80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:47.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.902+0000 7fabe1cff640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabc80a2c30 0x7fabc80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:47.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.902+0000 7fabd2ffd640 1 -- 172.21.15.112:0/3811448228 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fabcc0cf920 con 0x7fabdc104c10 2024-04-24T23:14:47.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.903+0000 7fabe1cff640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabc80a2c30 0x7fabc80a50f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fabd4005c20 tx=0x7fabd4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:47.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:47.907+0000 7fabd2ffd640 1 -- 172.21.15.112:0/3811448228 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fabcc09c490 con 0x7fabdc104c10 2024-04-24T23:14:48.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:48.127+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fabdc062370 con 0x7fabdc104c10 2024-04-24T23:14:48.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:48 smithi112 ceph-mon[29186]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:48.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:48 smithi150 ceph-mon[32247]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:48.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:48 smithi187 ceph-mon[33292]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:14:49.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.029+0000 7fabd2ffd640 1 -- 172.21.15.112:0/3811448228 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fabcc09bd90 con 0x7fabdc104c10 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabc80a2c30 msgr2=0x7fabc80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabc80a2c30 0x7fabc80a50f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fabd4005c20 tx=0x7fabd4000bc0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc104c10 msgr2=0x7fabdc197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc104c10 0x7fabdc197fa0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00e9c0 tx=0x7fabcc00ee80 comp rx=0 tx=0).stop 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 shutdown_connections 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabc80a2c30 0x7fabc80a50f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabdc107540 0x7fabdc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc104c10 0x7fabdc197fa0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 --2- 172.21.15.112:0/3811448228 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabdc1022e0 0x7fabdc197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 >> 172.21.15.112:0/3811448228 conn(0x7fabdc0fbc10 msgr2=0x7fabdc102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 shutdown_connections 2024-04-24T23:14:49.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.032+0000 7fabe3f63640 1 -- 172.21.15.112:0/3811448228 wait complete. 2024-04-24T23:14:49.049 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-24T23:14:49.060 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-24T23:14:49.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:49.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 -- 172.21.15.112:0/1312906681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488100680 msgr2=0x7f1488100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 --2- 172.21.15.112:0/1312906681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488100680 0x7f1488100aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1478009900 tx=0x7f147802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:49.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 -- 172.21.15.112:0/1312906681 shutdown_connections 2024-04-24T23:14:49.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 --2- 172.21.15.112:0/1312906681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1488102230 0x7f1488108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 --2- 172.21.15.112:0/1312906681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488101870 0x7f1488101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 --2- 172.21.15.112:0/1312906681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488100680 0x7f1488100aa0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 -- 172.21.15.112:0/1312906681 >> 172.21.15.112:0/1312906681 conn(0x7f14880fbbf0 msgr2=0x7f14880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 -- 172.21.15.112:0/1312906681 shutdown_connections 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.169+0000 7f148d229640 1 -- 172.21.15.112:0/1312906681 wait complete. 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148d229640 1 Processor -- start 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148d229640 1 -- start start 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148d229640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1488100680 0x7f1488197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148d229640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488101870 0x7f14881980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148d229640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488102230 0x7f148819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148d229640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f148810c430 con 0x7f1488102230 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148d229640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f148810c2b0 con 0x7f1488100680 2024-04-24T23:14:49.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148d229640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f148810c5b0 con 0x7f1488101870 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488102230 0x7f148819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488102230 0x7f148819f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50516/0 (socket says 172.21.15.112:50516) 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148759e640 1 -- 172.21.15.112:0/3048953161 learned_addr learned my addr 172.21.15.112:0/3048953161 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f1486d9d640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1488100680 0x7f1488197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.170+0000 7f148659c640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488101870 0x7f14881980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148759e640 1 -- 172.21.15.112:0/3048953161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488101870 msgr2=0x7f14881980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148759e640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488101870 0x7f14881980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148759e640 1 -- 172.21.15.112:0/3048953161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1488100680 msgr2=0x7f1488197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148759e640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1488100680 0x7f1488197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148759e640 1 -- 172.21.15.112:0/3048953161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148819f790 con 0x7f1488102230 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148659c640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488101870 0x7f14881980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148759e640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488102230 0x7f148819f120 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f148000cce0 tx=0x7f148000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f146ffff640 1 -- 172.21.15.112:0/3048953161 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148000f450 con 0x7f1488102230 2024-04-24T23:14:49.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f146ffff640 1 -- 172.21.15.112:0/3048953161 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f148000faf0 con 0x7f1488102230 2024-04-24T23:14:49.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148819fa80 con 0x7f1488102230 2024-04-24T23:14:49.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f146ffff640 1 -- 172.21.15.112:0/3048953161 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480018cc0 con 0x7f1488102230 2024-04-24T23:14:49.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.171+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148819ff80 con 0x7f1488102230 2024-04-24T23:14:49.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.172+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14881a0210 con 0x7f1488102230 2024-04-24T23:14:49.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.173+0000 7f146ffff640 1 -- 172.21.15.112:0/3048953161 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f148000f5f0 con 0x7f1488102230 2024-04-24T23:14:49.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.174+0000 7f146ffff640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14700a2b90 0x7f14700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.174+0000 7f1486d9d640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14700a2b90 0x7f14700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.174+0000 7f146ffff640 1 -- 172.21.15.112:0/3048953161 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f14800cf9c0 con 0x7f1488102230 2024-04-24T23:14:49.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.174+0000 7f1486d9d640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14700a2b90 0x7f14700a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1478006010 tx=0x7f1478031660 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.178+0000 7f146ffff640 1 -- 172.21.15.112:0/3048953161 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14800d1e20 con 0x7f1488102230 2024-04-24T23:14:49.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:49 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3811448228' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T23:14:49.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:49 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3811448228' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T23:14:49.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.360+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14881a0480 con 0x7f1488102230 2024-04-24T23:14:49.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.362+0000 7f146ffff640 1 -- 172.21.15.112:0/3048953161 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7f1480014030 con 0x7f1488102230 2024-04-24T23:14:49.364 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:49.364 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14700a2b90 msgr2=0x7f14700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14700a2b90 0x7f14700a5050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1478006010 tx=0x7f1478031660 comp rx=0 tx=0).stop 2024-04-24T23:14:49.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488102230 msgr2=0x7f148819f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488102230 0x7f148819f120 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f148000cce0 tx=0x7f148000a620 comp rx=0 tx=0).stop 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 shutdown_connections 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14700a2b90 0x7f14700a5050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1488102230 0x7f148819f120 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1488101870 0x7f14881980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 --2- 172.21.15.112:0/3048953161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1488100680 0x7f1488197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.365+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 >> 172.21.15.112:0/3048953161 conn(0x7f14880fbbf0 msgr2=0x7f14880fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.366+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 shutdown_connections 2024-04-24T23:14:49.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.366+0000 7f148d229640 1 -- 172.21.15.112:0/3048953161 wait complete. 2024-04-24T23:14:49.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:14:49.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:14:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 -- 172.21.15.112:0/1890376834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 msgr2=0x7f7594106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 --2- 172.21.15.112:0/1890376834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 0x7f7594106f20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7584009900 tx=0x7f758402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 -- 172.21.15.112:0/1890376834 shutdown_connections 2024-04-24T23:14:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 --2- 172.21.15.112:0/1890376834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7594107460 0x7f75941098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 --2- 172.21.15.112:0/1890376834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 0x7f7594106f20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 --2- 172.21.15.112:0/1890376834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7594102200 0x7f75941045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 -- 172.21.15.112:0/1890376834 >> 172.21.15.112:0/1890376834 conn(0x7f75940fbbf0 msgr2=0x7f75940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 -- 172.21.15.112:0/1890376834 shutdown_connections 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.482+0000 7f759c7ee640 1 -- 172.21.15.112:0/1890376834 wait complete. 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759c7ee640 1 Processor -- start 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759c7ee640 1 -- start start 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759c7ee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7594102200 0x7f7594197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759c7ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 0x7f7594198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759c7ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7594107460 0x7f759419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759c7ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f759410d590 con 0x7f7594104b30 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759c7ee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f759410d410 con 0x7f7594102200 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759c7ee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f759410d710 con 0x7f7594107460 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f7599d89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 0x7f7594198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f7599d89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 0x7f7594198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50528/0 (socket says 172.21.15.112:50528) 2024-04-24T23:14:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f7599d89640 1 -- 172.21.15.112:0/2461278971 learned_addr learned my addr 172.21.15.112:0/2461278971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759a58a640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7594102200 0x7f7594197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f759ad8b640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7594107460 0x7f759419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f7599d89640 1 -- 172.21.15.112:0/2461278971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7594107460 msgr2=0x7f759419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f7599d89640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7594107460 0x7f759419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f7599d89640 1 -- 172.21.15.112:0/2461278971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7594102200 msgr2=0x7f7594197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f7599d89640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7594102200 0x7f7594197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.483+0000 7f7599d89640 1 -- 172.21.15.112:0/2461278971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759419f830 con 0x7f7594104b30 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.484+0000 7f7599d89640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 0x7f7594198200 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7584009230 tx=0x7f7584004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.484+0000 7f758b7fe640 1 -- 172.21.15.112:0/2461278971 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758403f030 con 0x7f7594104b30 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.484+0000 7f758b7fe640 1 -- 172.21.15.112:0/2461278971 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7584003d00 con 0x7f7594104b30 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.484+0000 7f758b7fe640 1 -- 172.21.15.112:0/2461278971 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584035070 con 0x7f7594104b30 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.484+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759419fac0 con 0x7f7594104b30 2024-04-24T23:14:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.484+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759419ffb0 con 0x7f7594104b30 2024-04-24T23:14:49.489 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:49 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3811448228' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-24T23:14:49.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.485+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75940fd6e0 con 0x7f7594104b30 2024-04-24T23:14:49.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.486+0000 7f758b7fe640 1 -- 172.21.15.112:0/2461278971 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7584042030 con 0x7f7594104b30 2024-04-24T23:14:49.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.487+0000 7f758b7fe640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75800a2b90 0x7f75800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.487+0000 7f759a58a640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75800a2b90 0x7f75800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.487+0000 7f758b7fe640 1 -- 172.21.15.112:0/2461278971 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f75840efcb0 con 0x7f7594104b30 2024-04-24T23:14:49.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.487+0000 7f759a58a640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75800a2b90 0x7f75800a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f758c005c40 tx=0x7f758c00a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.491+0000 7f758b7fe640 1 -- 172.21.15.112:0/2461278971 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f75840bc820 con 0x7f7594104b30 2024-04-24T23:14:49.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.666+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7594062340 con 0x7f7594104b30 2024-04-24T23:14:49.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.666+0000 7f758b7fe640 1 -- 172.21.15.112:0/2461278971 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f75840bc120 con 0x7f7594104b30 2024-04-24T23:14:49.670 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75800a2b90 msgr2=0x7f75800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75800a2b90 0x7f75800a5050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f758c005c40 tx=0x7f758c00a3b0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 msgr2=0x7f7594198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 0x7f7594198200 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7584009230 tx=0x7f7584004430 comp rx=0 tx=0).stop 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 shutdown_connections 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75800a2b90 0x7f75800a5050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7594107460 0x7f759419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7594104b30 0x7f7594198200 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 --2- 172.21.15.112:0/2461278971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7594102200 0x7f7594197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 >> 172.21.15.112:0/2461278971 conn(0x7f75940fbbf0 msgr2=0x7f7594102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 shutdown_connections 2024-04-24T23:14:49.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.669+0000 7f759c7ee640 1 -- 172.21.15.112:0/2461278971 wait complete. 2024-04-24T23:14:49.682 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:14:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 -- 172.21.15.112:0/1687584639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c1006a0 msgr2=0x7fa55c100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 --2- 172.21.15.112:0/1687584639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c1006a0 0x7fa55c100ac0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa550009900 tx=0x7fa55002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 -- 172.21.15.112:0/1687584639 shutdown_connections 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 --2- 172.21.15.112:0/1687584639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa55c102250 0x7fa55c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 --2- 172.21.15.112:0/1687584639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa55c101890 0x7fa55c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 --2- 172.21.15.112:0/1687584639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c1006a0 0x7fa55c100ac0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 -- 172.21.15.112:0/1687584639 >> 172.21.15.112:0/1687584639 conn(0x7fa55c0fbc30 msgr2=0x7fa55c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 -- 172.21.15.112:0/1687584639 shutdown_connections 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 -- 172.21.15.112:0/1687584639 wait complete. 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 Processor -- start 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.781+0000 7fa5629e9640 1 -- start start 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa5629e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa55c1006a0 0x7fa55c197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa5629e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c101890 0x7fa55c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa5629e9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa55c102250 0x7fa55c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa5629e9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c10c450 con 0x7fa55c101890 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa5629e9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c10c2d0 con 0x7fa55c102250 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa5629e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa55c10c5d0 con 0x7fa55c1006a0 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c101890 0x7fa55c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c101890 0x7fa55c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50542/0 (socket says 172.21.15.112:50542) 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 -- 172.21.15.112:0/3470694075 learned_addr learned my addr 172.21.15.112:0/3470694075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa560f86640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa55c102250 0x7fa55c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55bfff640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa55c1006a0 0x7fa55c197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 -- 172.21.15.112:0/3470694075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa55c1006a0 msgr2=0x7fa55c197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa55c1006a0 0x7fa55c197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 -- 172.21.15.112:0/3470694075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa55c102250 msgr2=0x7fa55c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa55c102250 0x7fa55c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 -- 172.21.15.112:0/3470694075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55c19f800 con 0x7fa55c101890 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55bfff640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa55c1006a0 0x7fa55c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:49.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa55b7fe640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c101890 0x7fa55c198120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00d950 tx=0x7fa54c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa5597fa640 1 -- 172.21.15.112:0/3470694075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c017070 con 0x7fa55c101890 2024-04-24T23:14:49.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.782+0000 7fa5597fa640 1 -- 172.21.15.112:0/3470694075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa54c009c30 con 0x7fa55c101890 2024-04-24T23:14:49.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.783+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55c19faf0 con 0x7fa55c101890 2024-04-24T23:14:49.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.783+0000 7fa5597fa640 1 -- 172.21.15.112:0/3470694075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c00f740 con 0x7fa55c101890 2024-04-24T23:14:49.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.783+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55c1a0040 con 0x7fa55c101890 2024-04-24T23:14:49.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.783+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa55c1a0210 con 0x7fa55c101890 2024-04-24T23:14:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.784+0000 7fa5597fa640 1 -- 172.21.15.112:0/3470694075 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa54c010a80 con 0x7fa55c101890 2024-04-24T23:14:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.785+0000 7fa5597fa640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5440a2b90 0x7fa5440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.785+0000 7fa55bfff640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5440a2b90 0x7fa5440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.785+0000 7fa5597fa640 1 -- 172.21.15.112:0/3470694075 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa54c0cf0c0 con 0x7fa55c101890 2024-04-24T23:14:49.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.786+0000 7fa55bfff640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5440a2b90 0x7fa5440a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa550006010 tx=0x7fa550031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:49.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.789+0000 7fa5597fa640 1 -- 172.21.15.112:0/3470694075 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa54c009de0 con 0x7fa55c101890 2024-04-24T23:14:49.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.966+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa55c1a05a0 con 0x7fa55c101890 2024-04-24T23:14:49.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.966+0000 7fa5597fa640 1 -- 172.21.15.112:0/3470694075 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fa54c09bc30 con 0x7fa55c101890 2024-04-24T23:14:49.967 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:49.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5440a2b90 msgr2=0x7fa5440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5440a2b90 0x7fa5440a5050 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa550006010 tx=0x7fa550031640 comp rx=0 tx=0).stop 2024-04-24T23:14:49.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c101890 msgr2=0x7fa55c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:49.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c101890 0x7fa55c198120 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00d950 tx=0x7fa54c00de10 comp rx=0 tx=0).stop 2024-04-24T23:14:49.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 shutdown_connections 2024-04-24T23:14:49.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa5440a2b90 0x7fa5440a5050 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa55c102250 0x7fa55c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa55c101890 0x7fa55c198120 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 --2- 172.21.15.112:0/3470694075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa55c1006a0 0x7fa55c197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:49.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 >> 172.21.15.112:0/3470694075 conn(0x7fa55c0fbc30 msgr2=0x7fa55c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:49.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 shutdown_connections 2024-04-24T23:14:49.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:49.968+0000 7fa5629e9640 1 -- 172.21.15.112:0/3470694075 wait complete. 2024-04-24T23:14:49.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.081+0000 7f1797bc1640 1 -- 172.21.15.112:0/1550785786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1790102a20 msgr2=0x7f1790102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.081+0000 7f1797bc1640 1 --2- 172.21.15.112:0/1550785786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1790102a20 0x7f1790102ea0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1780009930 tx=0x7f178002a420 comp rx=0 tx=0).stop 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.081+0000 7f1797bc1640 1 -- 172.21.15.112:0/1550785786 shutdown_connections 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.081+0000 7f1797bc1640 1 --2- 172.21.15.112:0/1550785786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17901033e0 0x7f1790109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.081+0000 7f1797bc1640 1 --2- 172.21.15.112:0/1550785786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1790102a20 0x7f1790102ea0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.081+0000 7f1797bc1640 1 --2- 172.21.15.112:0/1550785786 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17900ffa50 0x7f17900ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.081+0000 7f1797bc1640 1 -- 172.21.15.112:0/1550785786 >> 172.21.15.112:0/1550785786 conn(0x7f17900fb380 msgr2=0x7f17900fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 -- 172.21.15.112:0/1550785786 shutdown_connections 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 -- 172.21.15.112:0/1550785786 wait complete. 2024-04-24T23:14:50.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 Processor -- start 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 -- start start 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17900ffa50 0x7f1790197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1790102a20 0x7f1790198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901033e0 0x7f179019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f179010d560 con 0x7f17901033e0 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f179010d3e0 con 0x7f17900ffa50 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f1797bc1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f179010d6e0 con 0x7f1790102a20 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f179615e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901033e0 0x7f179019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f179615e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901033e0 0x7f179019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50556/0 (socket says 172.21.15.112:50556) 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f179615e640 1 -- 172.21.15.112:0/2521895930 learned_addr learned my addr 172.21.15.112:0/2521895930 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.082+0000 7f179595d640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17900ffa50 0x7f1790197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f179515c640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1790102a20 0x7f1790198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f179615e640 1 -- 172.21.15.112:0/2521895930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1790102a20 msgr2=0x7f1790198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f179615e640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1790102a20 0x7f1790198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f179615e640 1 -- 172.21.15.112:0/2521895930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17900ffa50 msgr2=0x7f1790197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f179615e640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17900ffa50 0x7f1790197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f179615e640 1 -- 172.21.15.112:0/2521895930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179019f740 con 0x7f17901033e0 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f179515c640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1790102a20 0x7f1790198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f179615e640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901033e0 0x7f179019f0d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f178800cce0 tx=0x7f178800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f1786ffd640 1 -- 172.21.15.112:0/2521895930 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178800f450 con 0x7f17901033e0 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f1786ffd640 1 -- 172.21.15.112:0/2521895930 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f178800faf0 con 0x7f17901033e0 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179019fa30 con 0x7f17901033e0 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f1786ffd640 1 -- 172.21.15.112:0/2521895930 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788018cc0 con 0x7f17901033e0 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.083+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179019ff30 con 0x7f17901033e0 2024-04-24T23:14:50.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.084+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1790108af0 con 0x7f17901033e0 2024-04-24T23:14:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.085+0000 7f1786ffd640 1 -- 172.21.15.112:0/2521895930 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f178800f5f0 con 0x7f17901033e0 2024-04-24T23:14:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.085+0000 7f1786ffd640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f177c0a2b90 0x7f177c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.086+0000 7f179595d640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f177c0a2b90 0x7f177c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.086+0000 7f1786ffd640 1 -- 172.21.15.112:0/2521895930 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f17880cf940 con 0x7f17901033e0 2024-04-24T23:14:50.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.086+0000 7f179595d640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f177c0a2b90 0x7f177c0a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1790069bc0 tx=0x7f178c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.090+0000 7f1786ffd640 1 -- 172.21.15.112:0/2521895930 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1788014030 con 0x7f17901033e0 2024-04-24T23:14:50.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.265+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1790062340 con 0x7f17901033e0 2024-04-24T23:14:50.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.265+0000 7f1786ffd640 1 -- 172.21.15.112:0/2521895930 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f178809c4b0 con 0x7f17901033e0 2024-04-24T23:14:50.266 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:14:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f177c0a2b90 msgr2=0x7f177c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f177c0a2b90 0x7f177c0a5050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1790069bc0 tx=0x7f178c005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901033e0 msgr2=0x7f179019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901033e0 0x7f179019f0d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f178800cce0 tx=0x7f178800a620 comp rx=0 tx=0).stop 2024-04-24T23:14:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 shutdown_connections 2024-04-24T23:14:50.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f177c0a2b90 0x7f177c0a5050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17901033e0 0x7f179019f0d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1790102a20 0x7f1790198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 --2- 172.21.15.112:0/2521895930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f17900ffa50 0x7f1790197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 >> 172.21.15.112:0/2521895930 conn(0x7f17900fb380 msgr2=0x7f1790105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 shutdown_connections 2024-04-24T23:14:50.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.267+0000 7f1797bc1640 1 -- 172.21.15.112:0/2521895930 wait complete. 2024-04-24T23:14:50.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:50 smithi112 ceph-mon[29186]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:14:50.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3811448228' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T23:14:50.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:50 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:14:50.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3048953161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:50.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2461278971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:50.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:50 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3470694075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:50.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:14:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:50 smithi150 ceph-mon[32247]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:14:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:50 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3811448228' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T23:14:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:50 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:14:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:50 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3048953161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:50 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2461278971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:50 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3470694075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 -- 172.21.15.112:0/864079457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 msgr2=0x7f84b8108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 --2- 172.21.15.112:0/864079457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 0x7f84b8108ae0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009ca0 tx=0x7f84b002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 -- 172.21.15.112:0/864079457 shutdown_connections 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 --2- 172.21.15.112:0/864079457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 0x7f84b8108ae0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 --2- 172.21.15.112:0/864079457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84b8101890 0x7f84b8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 --2- 172.21.15.112:0/864079457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b81006a0 0x7f84b8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 -- 172.21.15.112:0/864079457 >> 172.21.15.112:0/864079457 conn(0x7f84b80fbc30 msgr2=0x7f84b80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 -- 172.21.15.112:0/864079457 shutdown_connections 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 -- 172.21.15.112:0/864079457 wait complete. 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.381+0000 7f84be69e640 1 Processor -- start 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84be69e640 1 -- start start 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84be69e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b81006a0 0x7f84b806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84be69e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84b8101890 0x7f84b819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84be69e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 0x7f84b819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84be69e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810c440 con 0x7f84b8102250 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84be69e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810c2c0 con 0x7f84b81006a0 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84be69e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f84b810c5c0 con 0x7f84b8101890 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 0x7f84b819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 0x7f84b819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50568/0 (socket says 172.21.15.112:50568) 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 -- 172.21.15.112:0/2458788871 learned_addr learned my addr 172.21.15.112:0/2458788871 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84b7fff640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b81006a0 0x7f84b806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84b77fe640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84b8101890 0x7f84b819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 -- 172.21.15.112:0/2458788871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84b8101890 msgr2=0x7f84b819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84b8101890 0x7f84b819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 -- 172.21.15.112:0/2458788871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b81006a0 msgr2=0x7f84b806a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b81006a0 0x7f84b806a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 -- 172.21.15.112:0/2458788871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b819f6c0 con 0x7f84b8102250 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84b7fff640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b81006a0 0x7f84b806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.382+0000 7f84bcc3b640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 0x7f84b819ee00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f84b0031a40 tx=0x7f84b0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.383+0000 7f84b57fa640 1 -- 172.21.15.112:0/2458788871 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b002fbe0 con 0x7f84b8102250 2024-04-24T23:14:50.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.383+0000 7f84b57fa640 1 -- 172.21.15.112:0/2458788871 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f84b002fd80 con 0x7f84b8102250 2024-04-24T23:14:50.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.383+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b819f950 con 0x7f84b8102250 2024-04-24T23:14:50.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.383+0000 7f84b57fa640 1 -- 172.21.15.112:0/2458788871 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0046980 con 0x7f84b8102250 2024-04-24T23:14:50.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.383+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b819fea0 con 0x7f84b8102250 2024-04-24T23:14:50.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.384+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b81078f0 con 0x7f84b8102250 2024-04-24T23:14:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.385+0000 7f84b57fa640 1 -- 172.21.15.112:0/2458788871 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f84b0030ce0 con 0x7f84b8102250 2024-04-24T23:14:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.385+0000 7f84b57fa640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.385+0000 7f84b7fff640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.385+0000 7f84b57fa640 1 -- 172.21.15.112:0/2458788871 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f84b00f0ae0 con 0x7f84b8102250 2024-04-24T23:14:50.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.386+0000 7f84b7fff640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f84a00a2b90 0x7f84a00a5050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f84b81016f0 tx=0x7f84a800a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.389+0000 7f84b57fa640 1 -- 172.21.15.112:0/2458788871 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f84b00bd6e0 con 0x7f84b8102250 2024-04-24T23:14:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:50 smithi187 ceph-mon[33292]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:14:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:50 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3811448228' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-24T23:14:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:50 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:14:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:50 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3048953161' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:50 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2461278971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:50 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3470694075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:50.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.564+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f84b8062340 con 0x7f84b8102250 2024-04-24T23:14:50.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.564+0000 7f84b57fa640 1 -- 172.21.15.112:0/2458788871 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f84b00bcfe0 con 0x7f84b8102250 2024-04-24T23:14:50.565 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f84a00a2b90 msgr2=0x7f84a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f84a00a2b90 0x7f84a00a5050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f84b81016f0 tx=0x7f84a800a5e0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 msgr2=0x7f84b819ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 0x7f84b819ee00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f84b0031a40 tx=0x7f84b0031ee0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 shutdown_connections 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f84a00a2b90 0x7f84a00a5050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84b8102250 0x7f84b819ee00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f84b8101890 0x7f84b819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 --2- 172.21.15.112:0/2458788871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84b81006a0 0x7f84b806a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 >> 172.21.15.112:0/2458788871 conn(0x7f84b80fbc30 msgr2=0x7f84b8104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 shutdown_connections 2024-04-24T23:14:50.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.567+0000 7f84be69e640 1 -- 172.21.15.112:0/2458788871 wait complete. 2024-04-24T23:14:50.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:50.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 -- 172.21.15.112:0/3986324021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f40ffe20 msgr2=0x7f51f4100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 --2- 172.21.15.112:0/3986324021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f40ffe20 0x7f51f4100240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f51f000aa10 tx=0x7f51f002b720 comp rx=0 tx=0).stop 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 -- 172.21.15.112:0/3986324021 shutdown_connections 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 --2- 172.21.15.112:0/3986324021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f4101930 0x7f51f4108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 --2- 172.21.15.112:0/3986324021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f4100f70 0x7f51f41013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 --2- 172.21.15.112:0/3986324021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f40ffe20 0x7f51f4100240 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 -- 172.21.15.112:0/3986324021 >> 172.21.15.112:0/3986324021 conn(0x7f51f40fb3b0 msgr2=0x7f51f40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 -- 172.21.15.112:0/3986324021 shutdown_connections 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.675+0000 7f51fba6b640 1 -- 172.21.15.112:0/3986324021 wait complete. 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fba6b640 1 Processor -- start 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fba6b640 1 -- start start 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fba6b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f40ffe20 0x7f51f4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fba6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f4100f70 0x7f51f4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fba6b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f4101930 0x7f51f419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fba6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51f410b3a0 con 0x7f51f4100f70 2024-04-24T23:14:50.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fba6b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f51f410b220 con 0x7f51f40ffe20 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fba6b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51f410b520 con 0x7f51f4101930 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51f9006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f4100f70 0x7f51f4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51f9006640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f4100f70 0x7f51f4198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39560/0 (socket says 172.21.15.112:39560) 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51f9006640 1 -- 172.21.15.112:0/741353917 learned_addr learned my addr 172.21.15.112:0/741353917 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51fa008640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f4101930 0x7f51f419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.676+0000 7f51f9807640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f40ffe20 0x7f51f4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51f9006640 1 -- 172.21.15.112:0/741353917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f4101930 msgr2=0x7f51f419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51f9006640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f4101930 0x7f51f419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51f9006640 1 -- 172.21.15.112:0/741353917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f40ffe20 msgr2=0x7f51f4197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51f9006640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f40ffe20 0x7f51f4197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51f9006640 1 -- 172.21.15.112:0/741353917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f419f7e0 con 0x7f51f4100f70 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51f9006640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f4100f70 0x7f51f4198100 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e9c0 tx=0x7f51e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51eaffd640 1 -- 172.21.15.112:0/741353917 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e400cd80 con 0x7f51f4100f70 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51eaffd640 1 -- 172.21.15.112:0/741353917 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f51e40035f0 con 0x7f51f4100f70 2024-04-24T23:14:50.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f419fad0 con 0x7f51f4100f70 2024-04-24T23:14:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51eaffd640 1 -- 172.21.15.112:0/741353917 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4010670 con 0x7f51f4100f70 2024-04-24T23:14:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.677+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f419ffd0 con 0x7f51f4100f70 2024-04-24T23:14:50.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.678+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f41a0260 con 0x7f51f4100f70 2024-04-24T23:14:50.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.679+0000 7f51eaffd640 1 -- 172.21.15.112:0/741353917 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f51e40037a0 con 0x7f51f4100f70 2024-04-24T23:14:50.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.680+0000 7f51eaffd640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f51e00a2b90 0x7f51e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.680+0000 7f51f9807640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f51e00a2b90 0x7f51e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.680+0000 7f51eaffd640 1 -- 172.21.15.112:0/741353917 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f51e40cf910 con 0x7f51f4100f70 2024-04-24T23:14:50.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.680+0000 7f51f9807640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f51e00a2b90 0x7f51e00a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f51f0000c00 tx=0x7f51f0030750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.683+0000 7f51eaffd640 1 -- 172.21.15.112:0/741353917 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f51e409c480 con 0x7f51f4100f70 2024-04-24T23:14:50.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.862+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51f41a0430 con 0x7f51f4100f70 2024-04-24T23:14:50.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.863+0000 7f51eaffd640 1 -- 172.21.15.112:0/741353917 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7f51e409bd80 con 0x7f51f4100f70 2024-04-24T23:14:50.864 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:50.865 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:50.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f51e00a2b90 msgr2=0x7f51e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f51e00a2b90 0x7f51e00a5050 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f51f0000c00 tx=0x7f51f0030750 comp rx=0 tx=0).stop 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f4100f70 msgr2=0x7f51f4198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f4100f70 0x7f51f4198100 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e9c0 tx=0x7f51e400ee80 comp rx=0 tx=0).stop 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 shutdown_connections 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f51e00a2b90 0x7f51e00a5050 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51f4101930 0x7f51f419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51f4100f70 0x7f51f4198100 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 --2- 172.21.15.112:0/741353917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51f40ffe20 0x7f51f4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 >> 172.21.15.112:0/741353917 conn(0x7f51f40fb3b0 msgr2=0x7f51f4106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.865+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 shutdown_connections 2024-04-24T23:14:50.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.866+0000 7f51fba6b640 1 -- 172.21.15.112:0/741353917 wait complete. 2024-04-24T23:14:50.878 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:14:50.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 -- 172.21.15.112:0/2372626259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102990 msgr2=0x7f7148102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 --2- 172.21.15.112:0/2372626259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102990 0x7f7148102e10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7138009900 tx=0x7f713802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 -- 172.21.15.112:0/2372626259 shutdown_connections 2024-04-24T23:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 --2- 172.21.15.112:0/2372626259 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7148103350 0x7f7148109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 --2- 172.21.15.112:0/2372626259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102990 0x7f7148102e10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 --2- 172.21.15.112:0/2372626259 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71481017a0 0x7f7148101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 -- 172.21.15.112:0/2372626259 >> 172.21.15.112:0/2372626259 conn(0x7f71480fcd10 msgr2=0x7f71480ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 -- 172.21.15.112:0/2372626259 shutdown_connections 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 -- 172.21.15.112:0/2372626259 wait complete. 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 Processor -- start 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.979+0000 7f7150722640 1 -- start start 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f7150722640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71481017a0 0x7f7148197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f7150722640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7148102990 0x7f71481980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f7150722640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148103350 0x7f714819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f7150722640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f714810d550 con 0x7f71481017a0 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f7150722640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f714810d3d0 con 0x7f7148103350 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f7150722640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f714810d6d0 con 0x7f7148102990 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71481017a0 0x7f7148197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71481017a0 0x7f7148197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39570/0 (socket says 172.21.15.112:39570) 2024-04-24T23:14:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 -- 172.21.15.112:0/3844868037 learned_addr learned my addr 172.21.15.112:0/3844868037 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714dcbd640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7148102990 0x7f71481980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714ecbf640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148103350 0x7f714819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 -- 172.21.15.112:0/3844868037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7148102990 msgr2=0x7f71481980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7148102990 0x7f71481980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 -- 172.21.15.112:0/3844868037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148103350 msgr2=0x7f714819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148103350 0x7f714819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 -- 172.21.15.112:0/3844868037 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714819f7b0 con 0x7f71481017a0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f714e4be640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71481017a0 0x7f7148197b90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f714400c970 tx=0x7f714400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f713f7fe640 1 -- 172.21.15.112:0/3844868037 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144004220 con 0x7f71481017a0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f713f7fe640 1 -- 172.21.15.112:0/3844868037 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f71440043c0 con 0x7f71481017a0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f713f7fe640 1 -- 172.21.15.112:0/3844868037 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714400f6b0 con 0x7f71481017a0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714819faa0 con 0x7f71481017a0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.980+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714819ffc0 con 0x7f71481017a0 2024-04-24T23:14:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.981+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71480689b0 con 0x7f71481017a0 2024-04-24T23:14:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.982+0000 7f713f7fe640 1 -- 172.21.15.112:0/3844868037 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f71440037a0 con 0x7f71481017a0 2024-04-24T23:14:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.983+0000 7f713f7fe640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.983+0000 7f714dcbd640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.983+0000 7f713f7fe640 1 -- 172.21.15.112:0/3844868037 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f71440cf560 con 0x7f71481017a0 2024-04-24T23:14:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.983+0000 7f714dcbd640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7138000c00 tx=0x7f7138031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:50.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:50.987+0000 7f713f7fe640 1 -- 172.21.15.112:0/3844868037 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7144014030 con 0x7f71481017a0 2024-04-24T23:14:51.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.166+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f714802d1f0 con 0x7f71481017a0 2024-04-24T23:14:51.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.167+0000 7f713f7fe640 1 -- 172.21.15.112:0/3844868037 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f7144003e40 con 0x7f71481017a0 2024-04-24T23:14:51.168 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:14:51.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 msgr2=0x7f71340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7138000c00 tx=0x7f7138031640 comp rx=0 tx=0).stop 2024-04-24T23:14:51.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71481017a0 msgr2=0x7f7148197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71481017a0 0x7f7148197b90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f714400c970 tx=0x7f714400ce30 comp rx=0 tx=0).stop 2024-04-24T23:14:51.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 shutdown_connections 2024-04-24T23:14:51.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148103350 0x7f714819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7148102990 0x7f71481980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 --2- 172.21.15.112:0/3844868037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71481017a0 0x7f7148197b90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 >> 172.21.15.112:0/3844868037 conn(0x7f71480fcd10 msgr2=0x7f7148105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 shutdown_connections 2024-04-24T23:14:51.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.169+0000 7f7150722640 1 -- 172.21.15.112:0/3844868037 wait complete. 2024-04-24T23:14:51.180 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:51 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2521895930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:51.180 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:51 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2458788871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:51.180 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:51 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/741353917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:51.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 -- 172.21.15.112:0/2656944030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 msgr2=0x7efd141123b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 --2- 172.21.15.112:0/2656944030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 0x7efd141123b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efd04009930 tx=0x7efd0402a420 comp rx=0 tx=0).stop 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 -- 172.21.15.112:0/2656944030 shutdown_connections 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 --2- 172.21.15.112:0/2656944030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 0x7efd141123b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 --2- 172.21.15.112:0/2656944030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd141090a0 0x7efd1410fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 --2- 172.21.15.112:0/2656944030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14108740 0x7efd14108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 -- 172.21.15.112:0/2656944030 >> 172.21.15.112:0/2656944030 conn(0x7efd1406e5d0 msgr2=0x7efd14070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 -- 172.21.15.112:0/2656944030 shutdown_connections 2024-04-24T23:14:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 -- 172.21.15.112:0/2656944030 wait complete. 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 Processor -- start 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 -- start start 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14108740 0x7efd141a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd141090a0 0x7efd141a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.282+0000 7efd19529640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 0x7efd141a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd19529640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd14115d50 con 0x7efd1410ffc0 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd19529640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7efd14115bd0 con 0x7efd14108740 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd19529640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd14115ed0 con 0x7efd141090a0 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 0x7efd141a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 0x7efd141a7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39586/0 (socket says 172.21.15.112:39586) 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 -- 172.21.15.112:0/1187736225 learned_addr learned my addr 172.21.15.112:0/1187736225 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd12ffd640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14108740 0x7efd141a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd127fc640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd141090a0 0x7efd141a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 -- 172.21.15.112:0/1187736225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd141090a0 msgr2=0x7efd141a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd141090a0 0x7efd141a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 -- 172.21.15.112:0/1187736225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14108740 msgr2=0x7efd141a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14108740 0x7efd141a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 -- 172.21.15.112:0/1187736225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd141a7ff0 con 0x7efd1410ffc0 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd137fe640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 0x7efd141a7980 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efd04009900 tx=0x7efd040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efcfbfff640 1 -- 172.21.15.112:0/1187736225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0403f030 con 0x7efd1410ffc0 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efcfbfff640 1 -- 172.21.15.112:0/1187736225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7efd04003d00 con 0x7efd1410ffc0 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd141a8280 con 0x7efd1410ffc0 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efcfbfff640 1 -- 172.21.15.112:0/1187736225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04035070 con 0x7efd1410ffc0 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.283+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd141a8770 con 0x7efd1410ffc0 2024-04-24T23:14:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.284+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd141a8a90 con 0x7efd1410ffc0 2024-04-24T23:14:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.286+0000 7efcfbfff640 1 -- 172.21.15.112:0/1187736225 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efd04042030 con 0x7efd1410ffc0 2024-04-24T23:14:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.287+0000 7efcfbfff640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efcfc0a2b90 0x7efcfc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.287+0000 7efd12ffd640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efcfc0a2b90 0x7efcfc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.287+0000 7efcfbfff640 1 -- 172.21.15.112:0/1187736225 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7efd040efce0 con 0x7efd1410ffc0 2024-04-24T23:14:51.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.287+0000 7efd12ffd640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efcfc0a2b90 0x7efcfc0a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.290+0000 7efcfbfff640 1 -- 172.21.15.112:0/1187736225 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efd040bc200 con 0x7efd1410ffc0 2024-04-24T23:14:51.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:51 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2521895930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:51.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:51 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2458788871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:51.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:51 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/741353917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.463+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efd141a8e20 con 0x7efd1410ffc0 2024-04-24T23:14:51.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.463+0000 7efcfbfff640 1 -- 172.21.15.112:0/1187736225 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7efd040c0500 con 0x7efd1410ffc0 2024-04-24T23:14:51.464 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:51.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efcfc0a2b90 msgr2=0x7efcfc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efcfc0a2b90 0x7efcfc0a5050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005fd0 tx=0x7efd0c009680 comp rx=0 tx=0).stop 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 msgr2=0x7efd141a7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 0x7efd141a7980 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efd04009900 tx=0x7efd040042f0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 shutdown_connections 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7efcfc0a2b90 0x7efcfc0a5050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd1410ffc0 0x7efd141a7980 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd141090a0 0x7efd141a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 --2- 172.21.15.112:0/1187736225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd14108740 0x7efd141a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 >> 172.21.15.112:0/1187736225 conn(0x7efd1406e5d0 msgr2=0x7efd14070230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 shutdown_connections 2024-04-24T23:14:51.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.465+0000 7efd19529640 1 -- 172.21.15.112:0/1187736225 wait complete. 2024-04-24T23:14:51.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:14:51.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:51 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2521895930' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:51.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:51 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2458788871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:51.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:51 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/741353917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 -- 172.21.15.112:0/4282974829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 msgr2=0x7f7cac101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/4282974829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 0x7f7cac101d10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009900 tx=0x7f7c9402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:51.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 -- 172.21.15.112:0/4282974829 shutdown_connections 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/4282974829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac102250 0x7f7cac108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/4282974829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 0x7f7cac101d10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/4282974829 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cac1006a0 0x7f7cac100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 -- 172.21.15.112:0/4282974829 >> 172.21.15.112:0/4282974829 conn(0x7f7cac0fbc30 msgr2=0x7f7cac0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 -- 172.21.15.112:0/4282974829 shutdown_connections 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.577+0000 7f7cb0b63640 1 -- 172.21.15.112:0/4282974829 wait complete. 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7cb0b63640 1 Processor -- start 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7cb0b63640 1 -- start start 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7cb0b63640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac1006a0 0x7f7cac199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7cb0b63640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 0x7f7cac19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7cb0b63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cac102250 0x7f7cac1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7cb0b63640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac10c450 con 0x7f7cac101890 2024-04-24T23:14:51.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7cb0b63640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac10c2d0 con 0x7f7cac1006a0 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7cb0b63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7cac10c5d0 con 0x7f7cac102250 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 0x7f7cac19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7ca3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 0x7f7cac19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39596/0 (socket says 172.21.15.112:39596) 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7ca3fff640 1 -- 172.21.15.112:0/2048454935 learned_addr learned my addr 172.21.15.112:0/2048454935 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7caa59c640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac1006a0 0x7f7cac199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7ca3fff640 1 -- 172.21.15.112:0/2048454935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cac102250 msgr2=0x7f7cac1a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7ca3fff640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cac102250 0x7f7cac1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7ca3fff640 1 -- 172.21.15.112:0/2048454935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac1006a0 msgr2=0x7f7cac199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7ca3fff640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac1006a0 0x7f7cac199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.578+0000 7f7ca3fff640 1 -- 172.21.15.112:0/2048454935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cac1a1960 con 0x7f7cac101890 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.579+0000 7f7ca3fff640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 0x7f7cac19a280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009a20 tx=0x7f7c94004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.579+0000 7f7ca37fe640 1 -- 172.21.15.112:0/2048454935 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9403f030 con 0x7f7cac101890 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.579+0000 7f7ca37fe640 1 -- 172.21.15.112:0/2048454935 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7c94003d00 con 0x7f7cac101890 2024-04-24T23:14:51.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.579+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cac1a1bf0 con 0x7f7cac101890 2024-04-24T23:14:51.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.579+0000 7f7ca37fe640 1 -- 172.21.15.112:0/2048454935 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94035070 con 0x7f7cac101890 2024-04-24T23:14:51.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.579+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cac1a20e0 con 0x7f7cac101890 2024-04-24T23:14:51.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.580+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cac1a22b0 con 0x7f7cac101890 2024-04-24T23:14:51.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.581+0000 7f7ca37fe640 1 -- 172.21.15.112:0/2048454935 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7c94042030 con 0x7f7cac101890 2024-04-24T23:14:51.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.581+0000 7f7ca37fe640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7c980a2b90 0x7f7c980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.581+0000 7f7caa59c640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7c980a2b90 0x7f7c980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.581+0000 7f7ca37fe640 1 -- 172.21.15.112:0/2048454935 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f7c940efbe0 con 0x7f7cac101890 2024-04-24T23:14:51.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.582+0000 7f7caa59c640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7c980a2b90 0x7f7c980a5050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005fd0 tx=0x7f7c9c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.586+0000 7f7ca37fe640 1 -- 172.21.15.112:0/2048454935 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7c940bc750 con 0x7f7cac101890 2024-04-24T23:14:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.761+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7cac1a2480 con 0x7f7cac101890 2024-04-24T23:14:51.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.761+0000 7f7ca37fe640 1 -- 172.21.15.112:0/2048454935 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f7c940bc050 con 0x7f7cac101890 2024-04-24T23:14:51.763 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:14:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7c980a2b90 msgr2=0x7f7c980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7c980a2b90 0x7f7c980a5050 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005fd0 tx=0x7f7c9c005850 comp rx=0 tx=0).stop 2024-04-24T23:14:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 msgr2=0x7f7cac19a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 0x7f7cac19a280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009a20 tx=0x7f7c94004430 comp rx=0 tx=0).stop 2024-04-24T23:14:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 shutdown_connections 2024-04-24T23:14:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7c980a2b90 0x7f7c980a5050 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7cac102250 0x7f7cac1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cac101890 0x7f7cac19a280 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 --2- 172.21.15.112:0/2048454935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cac1006a0 0x7f7cac199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 >> 172.21.15.112:0/2048454935 conn(0x7f7cac0fbc30 msgr2=0x7f7cac0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 shutdown_connections 2024-04-24T23:14:51.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.763+0000 7f7cb0b63640 1 -- 172.21.15.112:0/2048454935 wait complete. 2024-04-24T23:14:51.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:14:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 -- 172.21.15.112:0/2413521215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 msgr2=0x7f7148108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2413521215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 0x7f7148108ae0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7140009ca0 tx=0x7f714002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 -- 172.21.15.112:0/2413521215 shutdown_connections 2024-04-24T23:14:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2413521215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 0x7f7148108ae0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2413521215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7148101890 0x7f7148101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2413521215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71481006a0 0x7f7148100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 -- 172.21.15.112:0/2413521215 >> 172.21.15.112:0/2413521215 conn(0x7f71480fbc30 msgr2=0x7f71480fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 -- 172.21.15.112:0/2413521215 shutdown_connections 2024-04-24T23:14:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.876+0000 7f714fbf9640 1 -- 172.21.15.112:0/2413521215 wait complete. 2024-04-24T23:14:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714fbf9640 1 Processor -- start 2024-04-24T23:14:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714fbf9640 1 -- start start 2024-04-24T23:14:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714fbf9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71481006a0 0x7f71481969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714fbf9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148101890 0x7f7148196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714fbf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 0x7f714819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714fbf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f714810c450 con 0x7f7148102250 2024-04-24T23:14:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714fbf9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f714810c2d0 con 0x7f7148101890 2024-04-24T23:14:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714fbf9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f714810c5d0 con 0x7f71481006a0 2024-04-24T23:14:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 0x7f714819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 0x7f714819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39618/0 (socket says 172.21.15.112:39618) 2024-04-24T23:14:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 -- 172.21.15.112:0/2089425401 learned_addr learned my addr 172.21.15.112:0/2089425401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:51.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714d995640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71481006a0 0x7f71481969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714d194640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148101890 0x7f7148196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 -- 172.21.15.112:0/2089425401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71481006a0 msgr2=0x7f71481969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71481006a0 0x7f71481969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 -- 172.21.15.112:0/2089425401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148101890 msgr2=0x7f7148196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148101890 0x7f7148196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 -- 172.21.15.112:0/2089425401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714819e5c0 con 0x7f7148102250 2024-04-24T23:14:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714d194640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148101890 0x7f7148196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:51.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.877+0000 7f714e196640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 0x7f714819df50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7140031a40 tx=0x7f7140031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.878+0000 7f713effd640 1 -- 172.21.15.112:0/2089425401 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714002fbe0 con 0x7f7148102250 2024-04-24T23:14:51.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.878+0000 7f713effd640 1 -- 172.21.15.112:0/2089425401 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f714002fd80 con 0x7f7148102250 2024-04-24T23:14:51.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.878+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714819e850 con 0x7f7148102250 2024-04-24T23:14:51.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.878+0000 7f713effd640 1 -- 172.21.15.112:0/2089425401 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140046980 con 0x7f7148102250 2024-04-24T23:14:51.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.878+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714819eda0 con 0x7f7148102250 2024-04-24T23:14:51.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.879+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71481078f0 con 0x7f7148102250 2024-04-24T23:14:51.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.880+0000 7f713effd640 1 -- 172.21.15.112:0/2089425401 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7140030ce0 con 0x7f7148102250 2024-04-24T23:14:51.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.880+0000 7f713effd640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:51.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.880+0000 7f714d995640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:51.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.881+0000 7f713effd640 1 -- 172.21.15.112:0/2089425401 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f71400f0a60 con 0x7f7148102250 2024-04-24T23:14:51.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.881+0000 7f714d995640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f71481016f0 tx=0x7f7144005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:51.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:51.885+0000 7f713effd640 1 -- 172.21.15.112:0/2089425401 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f71400bd660 con 0x7f7148102250 2024-04-24T23:14:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.068+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7148062340 con 0x7f7148102250 2024-04-24T23:14:52.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.068+0000 7f713effd640 1 -- 172.21.15.112:0/2089425401 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f71400bcf60 con 0x7f7148102250 2024-04-24T23:14:52.069 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 msgr2=0x7f71340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f71481016f0 tx=0x7f7144005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 msgr2=0x7f714819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 0x7f714819df50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7140031a40 tx=0x7f7140031ee0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 shutdown_connections 2024-04-24T23:14:52.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f71340a2b90 0x7f71340a5050 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7148102250 0x7f714819df50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7148101890 0x7f7148196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 --2- 172.21.15.112:0/2089425401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71481006a0 0x7f71481969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 >> 172.21.15.112:0/2089425401 conn(0x7f71480fbc30 msgr2=0x7f7148104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 shutdown_connections 2024-04-24T23:14:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.070+0000 7f714fbf9640 1 -- 172.21.15.112:0/2089425401 wait complete. 2024-04-24T23:14:52.082 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-24T23:14:52.083 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:52.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2114000645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094102240 msgr2=0x7fe094108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2114000645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094102240 0x7fe094108ad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009ca0 tx=0x7fe08c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2114000645 shutdown_connections 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2114000645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094102240 0x7fe094108ad0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2114000645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe094101880 0x7fe094101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2114000645 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe094100690 0x7fe094100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2114000645 >> 172.21.15.112:0/2114000645 conn(0x7fe0940fbc40 msgr2=0x7fe0940fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2114000645 shutdown_connections 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2114000645 wait complete. 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.180+0000 7fe09a5a9640 1 Processor -- start 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe09a5a9640 1 -- start start 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe09a5a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094100690 0x7fe094197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe09a5a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe094101880 0x7fe094198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe09a5a9640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe094102240 0x7fe09419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe09a5a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe09410c440 con 0x7fe094100690 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe09a5a9640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe09410c2c0 con 0x7fe094102240 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe09a5a9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe09410c5c0 con 0x7fe094101880 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094100690 0x7fe094197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094100690 0x7fe094197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39638/0 (socket says 172.21.15.112:39638) 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 -- 172.21.15.112:0/2617604532 learned_addr learned my addr 172.21.15.112:0/2617604532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe0937fe640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe094101880 0x7fe094198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe098b46640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe094102240 0x7fe09419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 -- 172.21.15.112:0/2617604532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe094101880 msgr2=0x7fe094198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe094101880 0x7fe094198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 -- 172.21.15.112:0/2617604532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe094102240 msgr2=0x7fe09419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe094102240 0x7fe09419f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 -- 172.21.15.112:0/2617604532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09419f710 con 0x7fe094100690 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe0937fe640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe094101880 0x7fe094198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.181+0000 7fe093fff640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094100690 0x7fe094197af0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe08400d770 tx=0x7fe08400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.182+0000 7fe0917fa640 1 -- 172.21.15.112:0/2617604532 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084017070 con 0x7fe094100690 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.182+0000 7fe0917fa640 1 -- 172.21.15.112:0/2617604532 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe084009c30 con 0x7fe094100690 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.182+0000 7fe0917fa640 1 -- 172.21.15.112:0/2617604532 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084003e40 con 0x7fe094100690 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.182+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09419fa00 con 0x7fe094100690 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.182+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09419ff50 con 0x7fe094100690 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.183+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0941a0180 con 0x7fe094100690 2024-04-24T23:14:52.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.184+0000 7fe0917fa640 1 -- 172.21.15.112:0/2617604532 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe084004000 con 0x7fe094100690 2024-04-24T23:14:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.185+0000 7fe0917fa640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe07c0a2b90 0x7fe07c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.185+0000 7fe0937fe640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe07c0a2b90 0x7fe07c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.185+0000 7fe0917fa640 1 -- 172.21.15.112:0/2617604532 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fe0840cf2d0 con 0x7fe094100690 2024-04-24T23:14:52.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.185+0000 7fe0937fe640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe07c0a2b90 0x7fe07c0a5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe094199000 tx=0x7fe088005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.188+0000 7fe0917fa640 1 -- 172.21.15.112:0/2617604532 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe084009de0 con 0x7fe094100690 2024-04-24T23:14:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:52 smithi150 ceph-mon[32247]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:52 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3844868037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:52 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1187736225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:52 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2048454935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.400+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0941a0350 con 0x7fe094100690 2024-04-24T23:14:52.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.401+0000 7fe0917fa640 1 -- 172.21.15.112:0/2617604532 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7fe08409bde0 con 0x7fe094100690 2024-04-24T23:14:52.403 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:52.403 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:14:48.128683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24424,24425,24490,24524,24482]},"id":1}]} 2024-04-24T23:14:52.403 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-04-24T23:14:52.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe07c0a2b90 msgr2=0x7fe07c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe07c0a2b90 0x7fe07c0a5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe094199000 tx=0x7fe088005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:52.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094100690 msgr2=0x7fe094197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094100690 0x7fe094197af0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe08400d770 tx=0x7fe08400dc30 comp rx=0 tx=0).stop 2024-04-24T23:14:52.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 shutdown_connections 2024-04-24T23:14:52.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe07c0a2b90 0x7fe07c0a5050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe094102240 0x7fe09419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe094101880 0x7fe094198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 --2- 172.21.15.112:0/2617604532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe094100690 0x7fe094197af0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 >> 172.21.15.112:0/2617604532 conn(0x7fe0940fbc40 msgr2=0x7fe094069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 shutdown_connections 2024-04-24T23:14:52.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.403+0000 7fe09a5a9640 1 -- 172.21.15.112:0/2617604532 wait complete. 2024-04-24T23:14:52.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:52 smithi112 ceph-mon[29186]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:52.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:52 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3844868037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.415 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:52 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1187736225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.415 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:52 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2048454935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.416 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-24T23:14:52.416 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-24T23:14:52.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:14:52.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:52 smithi187 ceph-mon[33292]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-24T23:14:52.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:52 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3844868037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:52 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1187736225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:52 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2048454935' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.516+0000 7fda99ee7640 1 -- 172.21.15.112:0/1432913766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 msgr2=0x7fda941020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.516+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1432913766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 0x7fda941020d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fda84009ca0 tx=0x7fda8402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.516+0000 7fda99ee7640 1 -- 172.21.15.112:0/1432913766 shutdown_connections 2024-04-24T23:14:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.516+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1432913766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda941012c0 0x7fda94108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.516+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1432913766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda94100900 0x7fda94100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.516+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1432913766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 0x7fda941020d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 -- 172.21.15.112:0/1432913766 >> 172.21.15.112:0/1432913766 conn(0x7fda940fb3b0 msgr2=0x7fda940fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 -- 172.21.15.112:0/1432913766 shutdown_connections 2024-04-24T23:14:52.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 -- 172.21.15.112:0/1432913766 wait complete. 2024-04-24T23:14:52.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 Processor -- start 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 -- start start 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda94100900 0x7fda94100380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda941012c0 0x7fda9419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 0x7fda9419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda9410c4b0 con 0x7fda94101cb0 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fda9410c330 con 0x7fda941012c0 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda99ee7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda9410c630 con 0x7fda94100900 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 0x7fda9419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.517+0000 7fda93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 0x7fda9419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39662/0 (socket says 172.21.15.112:39662) 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda93fff640 1 -- 172.21.15.112:0/1126290556 learned_addr learned my addr 172.21.15.112:0/1126290556 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda937fe640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda94100900 0x7fda94100380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda93fff640 1 -- 172.21.15.112:0/1126290556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda94100900 msgr2=0x7fda94100380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda92ffd640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda941012c0 0x7fda9419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda93fff640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda94100900 0x7fda94100380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda93fff640 1 -- 172.21.15.112:0/1126290556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda941012c0 msgr2=0x7fda9419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda93fff640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda941012c0 0x7fda9419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda93fff640 1 -- 172.21.15.112:0/1126290556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9419f7d0 con 0x7fda94101cb0 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda937fe640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda94100900 0x7fda94100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda92ffd640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda941012c0 0x7fda9419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda93fff640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 0x7fda9419ef10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c970 tx=0x7fda8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda90ff9640 1 -- 172.21.15.112:0/1126290556 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c007bb0 con 0x7fda94101cb0 2024-04-24T23:14:52.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9419fac0 con 0x7fda94101cb0 2024-04-24T23:14:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda90ff9640 1 -- 172.21.15.112:0/1126290556 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fda8c007d50 con 0x7fda94101cb0 2024-04-24T23:14:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda90ff9640 1 -- 172.21.15.112:0/1126290556 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c0188a0 con 0x7fda94101cb0 2024-04-24T23:14:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.518+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9419ffc0 con 0x7fda94101cb0 2024-04-24T23:14:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.519+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda941079e0 con 0x7fda94101cb0 2024-04-24T23:14:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.520+0000 7fda90ff9640 1 -- 172.21.15.112:0/1126290556 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fda8c018a80 con 0x7fda94101cb0 2024-04-24T23:14:52.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.521+0000 7fda90ff9640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fda7c0a2b90 0x7fda7c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.521+0000 7fda937fe640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fda7c0a2b90 0x7fda7c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.521+0000 7fda90ff9640 1 -- 172.21.15.112:0/1126290556 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fda8c0cf670 con 0x7fda94101cb0 2024-04-24T23:14:52.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.521+0000 7fda937fe640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fda7c0a2b90 0x7fda7c0a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fda84006010 tx=0x7fda84009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.524+0000 7fda90ff9640 1 -- 172.21.15.112:0/1126290556 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fda8c014030 con 0x7fda94101cb0 2024-04-24T23:14:52.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.698+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda94061b80 con 0x7fda7c0a2b90 2024-04-24T23:14:52.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.707+0000 7fda90ff9640 1 -- 172.21.15.112:0/1126290556 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340539 (secure 0 0 0) 0x7fda94061b80 con 0x7fda7c0a2b90 2024-04-24T23:14:52.710 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:52.715 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-24T23:14:52.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fda7c0a2b90 msgr2=0x7fda7c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fda7c0a2b90 0x7fda7c0a5050 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fda84006010 tx=0x7fda84009360 comp rx=0 tx=0).stop 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 msgr2=0x7fda9419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 0x7fda9419ef10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c970 tx=0x7fda8c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 shutdown_connections 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fda7c0a2b90 0x7fda7c0a5050 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda94101cb0 0x7fda9419ef10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda941012c0 0x7fda9419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 --2- 172.21.15.112:0/1126290556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda94100900 0x7fda94100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 >> 172.21.15.112:0/1126290556 conn(0x7fda940fb3b0 msgr2=0x7fda94104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 shutdown_connections 2024-04-24T23:14:52.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.711+0000 7fda99ee7640 1 -- 172.21.15.112:0/1126290556 wait complete. 2024-04-24T23:14:52.723 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-04-24T23:14:50.821528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661777,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1318,"num_read_kb":780,"num_write":437,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384776,"kb_used_data":14768,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557048,"statfs":{"total":1151940427776,"available":1151546417152,"internally_reserved":0,"allocated":15122432,"data_stored":8253369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":57,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003075"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515637+0000","last_change":"2024-04-24T23:12:28.013506+0000","last_active":"2024-04-24T23:14:29.515637+0000","last_peered":"2024-04-24T23:14:29.515637+0000","last_clean":"2024-04-24T23:14:29.515637+0000","last_became_active":"2024-04-24T23:12:28.012652+0000","last_became_peered":"2024-04-24T23:12:28.012652+0000","last_unstale":"2024-04-24T23:14:29.515637+0000","last_undegraded":"2024-04-24T23:14:29.515637+0000","last_fullsized":"2024-04-24T23:14:29.515637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:57.420465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296164+0000","last_change":"2024-04-24T23:12:28.004998+0000","last_active":"2024-04-24T23:14:29.296164+0000","last_peered":"2024-04-24T23:14:29.296164+0000","last_clean":"2024-04-24T23:14:29.296164+0000","last_became_active":"2024-04-24T23:12:28.004706+0000","last_became_peered":"2024-04-24T23:12:28.004706+0000","last_unstale":"2024-04-24T23:14:29.296164+0000","last_undegraded":"2024-04-24T23:14:29.296164+0000","last_fullsized":"2024-04-24T23:14:29.296164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.759515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296026+0000","last_change":"2024-04-24T23:12:28.019170+0000","last_active":"2024-04-24T23:14:29.296026+0000","last_peered":"2024-04-24T23:14:29.296026+0000","last_clean":"2024-04-24T23:14:29.296026+0000","last_became_active":"2024-04-24T23:12:28.018815+0000","last_became_peered":"2024-04-24T23:12:28.018815+0000","last_unstale":"2024-04-24T23:14:29.296026+0000","last_undegraded":"2024-04-24T23:14:29.296026+0000","last_fullsized":"2024-04-24T23:14:29.296026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:43.830598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009162+0000","last_change":"2024-04-24T23:12:28.008662+0000","last_active":"2024-04-24T23:14:29.009162+0000","last_peered":"2024-04-24T23:14:29.009162+0000","last_clean":"2024-04-24T23:14:29.009162+0000","last_became_active":"2024-04-24T23:12:28.008282+0000","last_became_peered":"2024-04-24T23:12:28.008282+0000","last_unstale":"2024-04-24T23:14:29.009162+0000","last_undegraded":"2024-04-24T23:14:29.009162+0000","last_fullsized":"2024-04-24T23:14:29.009162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:17.558899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931843+0000","last_change":"2024-04-24T23:12:28.005212+0000","last_active":"2024-04-24T23:14:28.931843+0000","last_peered":"2024-04-24T23:14:28.931843+0000","last_clean":"2024-04-24T23:14:28.931843+0000","last_became_active":"2024-04-24T23:12:28.004853+0000","last_became_peered":"2024-04-24T23:12:28.004853+0000","last_unstale":"2024-04-24T23:14:28.931843+0000","last_undegraded":"2024-04-24T23:14:28.931843+0000","last_fullsized":"2024-04-24T23:14:28.931843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:33.650922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931438+0000","last_change":"2024-04-24T23:12:28.004546+0000","last_active":"2024-04-24T23:14:28.931438+0000","last_peered":"2024-04-24T23:14:28.931438+0000","last_clean":"2024-04-24T23:14:28.931438+0000","last_became_active":"2024-04-24T23:12:28.004194+0000","last_became_peered":"2024-04-24T23:12:28.004194+0000","last_unstale":"2024-04-24T23:14:28.931438+0000","last_undegraded":"2024-04-24T23:14:28.931438+0000","last_fullsized":"2024-04-24T23:14:28.931438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:57.772271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008625+0000","last_change":"2024-04-24T23:12:28.009436+0000","last_active":"2024-04-24T23:14:29.008625+0000","last_peered":"2024-04-24T23:14:29.008625+0000","last_clean":"2024-04-24T23:14:29.008625+0000","last_became_active":"2024-04-24T23:12:28.008219+0000","last_became_peered":"2024-04-24T23:12:28.008219+0000","last_unstale":"2024-04-24T23:14:29.008625+0000","last_undegraded":"2024-04-24T23:14:29.008625+0000","last_fullsized":"2024-04-24T23:14:29.008625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930928+0000","last_change":"2024-04-24T23:13:27.957345+0000","last_active":"2024-04-24T23:14:28.930928+0000","last_peered":"2024-04-24T23:14:28.930928+0000","last_clean":"2024-04-24T23:14:28.930928+0000","last_became_active":"2024-04-24T23:13:27.956989+0000","last_became_peered":"2024-04-24T23:13:27.956989+0000","last_unstale":"2024-04-24T23:14:28.930928+0000","last_undegraded":"2024-04-24T23:14:28.930928+0000","last_fullsized":"2024-04-24T23:14:28.930928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:42.120100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515437+0000","last_change":"2024-04-24T23:12:28.004948+0000","last_active":"2024-04-24T23:14:29.515437+0000","last_peered":"2024-04-24T23:14:29.515437+0000","last_clean":"2024-04-24T23:14:29.515437+0000","last_became_active":"2024-04-24T23:12:28.004380+0000","last_became_peered":"2024-04-24T23:12:28.004380+0000","last_unstale":"2024-04-24T23:14:29.515437+0000","last_undegraded":"2024-04-24T23:14:29.515437+0000","last_fullsized":"2024-04-24T23:14:29.515437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:53.999210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931753+0000","last_change":"2024-04-24T23:12:28.005192+0000","last_active":"2024-04-24T23:14:28.931753+0000","last_peered":"2024-04-24T23:14:28.931753+0000","last_clean":"2024-04-24T23:14:28.931753+0000","last_became_active":"2024-04-24T23:12:28.004860+0000","last_became_peered":"2024-04-24T23:12:28.004860+0000","last_unstale":"2024-04-24T23:14:28.931753+0000","last_undegraded":"2024-04-24T23:14:28.931753+0000","last_fullsized":"2024-04-24T23:14:28.931753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:21.874408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009736+0000","last_change":"2024-04-24T23:12:28.016098+0000","last_active":"2024-04-24T23:14:29.009736+0000","last_peered":"2024-04-24T23:14:29.009736+0000","last_clean":"2024-04-24T23:14:29.009736+0000","last_became_active":"2024-04-24T23:12:28.015736+0000","last_became_peered":"2024-04-24T23:12:28.015736+0000","last_unstale":"2024-04-24T23:14:29.009736+0000","last_undegraded":"2024-04-24T23:14:29.009736+0000","last_fullsized":"2024-04-24T23:14:29.009736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.071797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009000+0000","last_change":"2024-04-24T23:12:28.010651+0000","last_active":"2024-04-24T23:14:29.009000+0000","last_peered":"2024-04-24T23:14:29.009000+0000","last_clean":"2024-04-24T23:14:29.009000+0000","last_became_active":"2024-04-24T23:12:28.009700+0000","last_became_peered":"2024-04-24T23:12:28.009700+0000","last_unstale":"2024-04-24T23:14:29.009000+0000","last_undegraded":"2024-04-24T23:14:29.009000+0000","last_fullsized":"2024-04-24T23:14:29.009000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:26.172386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.934326+0000","last_change":"2024-04-24T23:14:28.934326+0000","last_active":"2024-04-24T23:14:28.934326+0000","last_peered":"2024-04-24T23:14:28.934326+0000","last_clean":"2024-04-24T23:14:28.934326+0000","last_became_active":"2024-04-24T23:14:28.933888+0000","last_became_peered":"2024-04-24T23:14:28.933888+0000","last_unstale":"2024-04-24T23:14:28.934326+0000","last_undegraded":"2024-04-24T23:14:28.934326+0000","last_fullsized":"2024-04-24T23:14:28.934326+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:04.909941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295666+0000","last_change":"2024-04-24T23:12:28.004140+0000","last_active":"2024-04-24T23:14:29.295666+0000","last_peered":"2024-04-24T23:14:29.295666+0000","last_clean":"2024-04-24T23:14:29.295666+0000","last_became_active":"2024-04-24T23:12:28.003794+0000","last_became_peered":"2024-04-24T23:12:28.003794+0000","last_unstale":"2024-04-24T23:14:29.295666+0000","last_undegraded":"2024-04-24T23:14:29.295666+0000","last_fullsized":"2024-04-24T23:14:29.295666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:02.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295921+0000","last_change":"2024-04-24T23:12:28.006834+0000","last_active":"2024-04-24T23:14:29.295921+0000","last_peered":"2024-04-24T23:14:29.295921+0000","last_clean":"2024-04-24T23:14:29.295921+0000","last_became_active":"2024-04-24T23:12:28.006504+0000","last_became_peered":"2024-04-24T23:12:28.006504+0000","last_unstale":"2024-04-24T23:14:29.295921+0000","last_undegraded":"2024-04-24T23:14:29.295921+0000","last_fullsized":"2024-04-24T23:14:29.295921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:21.194464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.514939+0000","last_change":"2024-04-24T23:12:28.013132+0000","last_active":"2024-04-24T23:14:29.514939+0000","last_peered":"2024-04-24T23:14:29.514939+0000","last_clean":"2024-04-24T23:14:29.514939+0000","last_became_active":"2024-04-24T23:12:28.012751+0000","last_became_peered":"2024-04-24T23:12:28.012751+0000","last_unstale":"2024-04-24T23:14:29.514939+0000","last_undegraded":"2024-04-24T23:14:29.514939+0000","last_fullsized":"2024-04-24T23:14:29.514939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.942985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515426+0000","last_change":"2024-04-24T23:12:28.013719+0000","last_active":"2024-04-24T23:14:29.515426+0000","last_peered":"2024-04-24T23:14:29.515426+0000","last_clean":"2024-04-24T23:14:29.515426+0000","last_became_active":"2024-04-24T23:12:28.013108+0000","last_became_peered":"2024-04-24T23:12:28.013108+0000","last_unstale":"2024-04-24T23:14:29.515426+0000","last_undegraded":"2024-04-24T23:14:29.515426+0000","last_fullsized":"2024-04-24T23:14:29.515426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:38.248867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932637+0000","last_change":"2024-04-24T23:12:28.012386+0000","last_active":"2024-04-24T23:14:28.932637+0000","last_peered":"2024-04-24T23:14:28.932637+0000","last_clean":"2024-04-24T23:14:28.932637+0000","last_became_active":"2024-04-24T23:12:28.011340+0000","last_became_peered":"2024-04-24T23:12:28.011340+0000","last_unstale":"2024-04-24T23:14:28.932637+0000","last_undegraded":"2024-04-24T23:14:28.932637+0000","last_fullsized":"2024-04-24T23:14:28.932637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:48.115600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930262+0000","last_change":"2024-04-24T23:12:28.004033+0000","last_active":"2024-04-24T23:14:28.930262+0000","last_peered":"2024-04-24T23:14:28.930262+0000","last_clean":"2024-04-24T23:14:28.930262+0000","last_became_active":"2024-04-24T23:12:28.003648+0000","last_became_peered":"2024-04-24T23:12:28.003648+0000","last_unstale":"2024-04-24T23:14:28.930262+0000","last_undegraded":"2024-04-24T23:14:28.930262+0000","last_fullsized":"2024-04-24T23:14:28.930262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:13.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.297170+0000","last_change":"2024-04-24T23:14:29.297170+0000","last_active":"2024-04-24T23:14:29.297170+0000","last_peered":"2024-04-24T23:14:29.297170+0000","last_clean":"2024-04-24T23:14:29.297170+0000","last_became_active":"2024-04-24T23:14:29.296794+0000","last_became_peered":"2024-04-24T23:14:29.296794+0000","last_unstale":"2024-04-24T23:14:29.297170+0000","last_undegraded":"2024-04-24T23:14:29.297170+0000","last_fullsized":"2024-04-24T23:14:29.297170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:19:14.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":158,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:46.030498+0000","last_change":"2024-04-24T23:12:27.005325+0000","last_active":"2024-04-24T23:14:46.030498+0000","last_peered":"2024-04-24T23:14:46.030498+0000","last_clean":"2024-04-24T23:14:46.030498+0000","last_became_active":"2024-04-24T23:12:27.004627+0000","last_became_peered":"2024-04-24T23:12:27.004627+0000","last_unstale":"2024-04-24T23:14:46.030498+0000","last_undegraded":"2024-04-24T23:14:46.030498+0000","last_fullsized":"2024-04-24T23:14:46.030498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.985647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928981+0000","last_change":"2024-04-24T23:12:28.007872+0000","last_active":"2024-04-24T23:14:28.928981+0000","last_peered":"2024-04-24T23:14:28.928981+0000","last_clean":"2024-04-24T23:14:28.928981+0000","last_became_active":"2024-04-24T23:12:28.007493+0000","last_became_peered":"2024-04-24T23:12:28.007493+0000","last_unstale":"2024-04-24T23:14:28.928981+0000","last_undegraded":"2024-04-24T23:14:28.928981+0000","last_fullsized":"2024-04-24T23:14:28.928981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:26.686831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008190+0000","last_change":"2024-04-24T23:12:32.031138+0000","last_active":"2024-04-24T23:14:29.008190+0000","last_peered":"2024-04-24T23:14:29.008190+0000","last_clean":"2024-04-24T23:14:29.008190+0000","last_became_active":"2024-04-24T23:12:32.030626+0000","last_became_peered":"2024-04-24T23:12:32.030626+0000","last_unstale":"2024-04-24T23:14:29.008190+0000","last_undegraded":"2024-04-24T23:14:29.008190+0000","last_fullsized":"2024-04-24T23:14:29.008190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929177+0000","last_change":"2024-04-24T23:12:26.999954+0000","last_active":"2024-04-24T23:14:28.929177+0000","last_peered":"2024-04-24T23:14:28.929177+0000","last_clean":"2024-04-24T23:14:28.929177+0000","last_became_active":"2024-04-24T23:12:26.999507+0000","last_became_peered":"2024-04-24T23:12:26.999507+0000","last_unstale":"2024-04-24T23:14:28.929177+0000","last_undegraded":"2024-04-24T23:14:28.929177+0000","last_fullsized":"2024-04-24T23:14:28.929177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:23.984002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296210+0000","last_change":"2024-04-24T23:12:28.019342+0000","last_active":"2024-04-24T23:14:29.296210+0000","last_peered":"2024-04-24T23:14:29.296210+0000","last_clean":"2024-04-24T23:14:29.296210+0000","last_became_active":"2024-04-24T23:12:28.018976+0000","last_became_peered":"2024-04-24T23:12:28.018976+0000","last_unstale":"2024-04-24T23:14:29.296210+0000","last_undegraded":"2024-04-24T23:14:29.296210+0000","last_fullsized":"2024-04-24T23:14:29.296210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:18.737495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008391+0000","last_change":"2024-04-24T23:12:32.031015+0000","last_active":"2024-04-24T23:14:29.008391+0000","last_peered":"2024-04-24T23:14:29.008391+0000","last_clean":"2024-04-24T23:14:29.008391+0000","last_became_active":"2024-04-24T23:12:32.030564+0000","last_became_peered":"2024-04-24T23:12:32.030564+0000","last_unstale":"2024-04-24T23:14:29.008391+0000","last_undegraded":"2024-04-24T23:14:29.008391+0000","last_fullsized":"2024-04-24T23:14:29.008391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:12.247397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931822+0000","last_change":"2024-04-24T23:12:27.001229+0000","last_active":"2024-04-24T23:14:28.931822+0000","last_peered":"2024-04-24T23:14:28.931822+0000","last_clean":"2024-04-24T23:14:28.931822+0000","last_became_active":"2024-04-24T23:12:27.000889+0000","last_became_peered":"2024-04-24T23:12:27.000889+0000","last_unstale":"2024-04-24T23:14:28.931822+0000","last_undegraded":"2024-04-24T23:14:28.931822+0000","last_fullsized":"2024-04-24T23:14:28.931822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:39.189828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515092+0000","last_change":"2024-04-24T23:12:28.012309+0000","last_active":"2024-04-24T23:14:29.515092+0000","last_peered":"2024-04-24T23:14:29.515092+0000","last_clean":"2024-04-24T23:14:29.515092+0000","last_became_active":"2024-04-24T23:12:28.011537+0000","last_became_peered":"2024-04-24T23:12:28.011537+0000","last_unstale":"2024-04-24T23:14:29.515092+0000","last_undegraded":"2024-04-24T23:14:29.515092+0000","last_fullsized":"2024-04-24T23:14:29.515092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:38.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008237+0000","last_change":"2024-04-24T23:12:32.030830+0000","last_active":"2024-04-24T23:14:29.008237+0000","last_peered":"2024-04-24T23:14:29.008237+0000","last_clean":"2024-04-24T23:14:29.008237+0000","last_became_active":"2024-04-24T23:12:32.030318+0000","last_became_peered":"2024-04-24T23:12:32.030318+0000","last_unstale":"2024-04-24T23:14:29.008237+0000","last_undegraded":"2024-04-24T23:14:29.008237+0000","last_fullsized":"2024-04-24T23:14:29.008237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:25.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931622+0000","last_change":"2024-04-24T23:12:26.997926+0000","last_active":"2024-04-24T23:14:28.931622+0000","last_peered":"2024-04-24T23:14:28.931622+0000","last_clean":"2024-04-24T23:14:28.931622+0000","last_became_active":"2024-04-24T23:12:26.997552+0000","last_became_peered":"2024-04-24T23:12:26.997552+0000","last_unstale":"2024-04-24T23:14:28.931622+0000","last_undegraded":"2024-04-24T23:14:28.931622+0000","last_fullsized":"2024-04-24T23:14:28.931622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:50.926624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012647+0000","last_change":"2024-04-24T23:13:28.338152+0000","last_active":"2024-04-24T23:14:29.012647+0000","last_peered":"2024-04-24T23:14:29.012647+0000","last_clean":"2024-04-24T23:14:29.012647+0000","last_became_active":"2024-04-24T23:13:28.337728+0000","last_became_peered":"2024-04-24T23:13:28.337728+0000","last_unstale":"2024-04-24T23:14:29.012647+0000","last_undegraded":"2024-04-24T23:14:29.012647+0000","last_fullsized":"2024-04-24T23:14:29.012647+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.179108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929875+0000","last_change":"2024-04-24T23:12:32.021601+0000","last_active":"2024-04-24T23:14:28.929875+0000","last_peered":"2024-04-24T23:14:28.929875+0000","last_clean":"2024-04-24T23:14:28.929875+0000","last_became_active":"2024-04-24T23:12:32.021141+0000","last_became_peered":"2024-04-24T23:12:32.021141+0000","last_unstale":"2024-04-24T23:14:28.929875+0000","last_undegraded":"2024-04-24T23:14:28.929875+0000","last_fullsized":"2024-04-24T23:14:28.929875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:53.795337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'8","reported_seq":161,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.044736+0000","last_change":"2024-04-24T23:12:27.002639+0000","last_active":"2024-04-24T23:14:49.044736+0000","last_peered":"2024-04-24T23:14:49.044736+0000","last_clean":"2024-04-24T23:14:49.044736+0000","last_became_active":"2024-04-24T23:12:27.002295+0000","last_became_peered":"2024-04-24T23:12:27.002295+0000","last_unstale":"2024-04-24T23:14:49.044736+0000","last_undegraded":"2024-04-24T23:14:49.044736+0000","last_fullsized":"2024-04-24T23:14:49.044736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:40.962878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931586+0000","last_change":"2024-04-24T23:12:28.000123+0000","last_active":"2024-04-24T23:14:28.931586+0000","last_peered":"2024-04-24T23:14:28.931586+0000","last_clean":"2024-04-24T23:14:28.931586+0000","last_became_active":"2024-04-24T23:12:27.999778+0000","last_became_peered":"2024-04-24T23:12:27.999778+0000","last_unstale":"2024-04-24T23:14:28.931586+0000","last_undegraded":"2024-04-24T23:14:28.931586+0000","last_fullsized":"2024-04-24T23:14:28.931586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:03.827336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929917+0000","last_change":"2024-04-24T23:12:32.023886+0000","last_active":"2024-04-24T23:14:28.929917+0000","last_peered":"2024-04-24T23:14:28.929917+0000","last_clean":"2024-04-24T23:14:28.929917+0000","last_became_active":"2024-04-24T23:12:32.022796+0000","last_became_peered":"2024-04-24T23:12:32.022796+0000","last_unstale":"2024-04-24T23:14:28.929917+0000","last_undegraded":"2024-04-24T23:14:28.929917+0000","last_fullsized":"2024-04-24T23:14:28.929917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:17.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931226+0000","last_change":"2024-04-24T23:12:26.999210+0000","last_active":"2024-04-24T23:14:28.931226+0000","last_peered":"2024-04-24T23:14:28.931226+0000","last_clean":"2024-04-24T23:14:28.931226+0000","last_became_active":"2024-04-24T23:12:26.998642+0000","last_became_peered":"2024-04-24T23:12:26.998642+0000","last_unstale":"2024-04-24T23:14:28.931226+0000","last_undegraded":"2024-04-24T23:14:28.931226+0000","last_fullsized":"2024-04-24T23:14:28.931226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:39.171955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012866+0000","last_change":"2024-04-24T23:13:27.990737+0000","last_active":"2024-04-24T23:14:29.012866+0000","last_peered":"2024-04-24T23:14:29.012866+0000","last_clean":"2024-04-24T23:14:29.012866+0000","last_became_active":"2024-04-24T23:13:27.990428+0000","last_became_peered":"2024-04-24T23:13:27.990428+0000","last_unstale":"2024-04-24T23:14:29.012866+0000","last_undegraded":"2024-04-24T23:14:29.012866+0000","last_fullsized":"2024-04-24T23:14:29.012866+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:40.163241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515481+0000","last_change":"2024-04-24T23:12:32.028192+0000","last_active":"2024-04-24T23:14:29.515481+0000","last_peered":"2024-04-24T23:14:29.515481+0000","last_clean":"2024-04-24T23:14:29.515481+0000","last_became_active":"2024-04-24T23:12:32.027715+0000","last_became_peered":"2024-04-24T23:12:32.027715+0000","last_unstale":"2024-04-24T23:14:29.515481+0000","last_undegraded":"2024-04-24T23:14:29.515481+0000","last_fullsized":"2024-04-24T23:14:29.515481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:22.577980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931020+0000","last_change":"2024-04-24T23:12:27.000212+0000","last_active":"2024-04-24T23:14:28.931020+0000","last_peered":"2024-04-24T23:14:28.931020+0000","last_clean":"2024-04-24T23:14:28.931020+0000","last_became_active":"2024-04-24T23:12:26.999844+0000","last_became_peered":"2024-04-24T23:12:26.999844+0000","last_unstale":"2024-04-24T23:14:28.931020+0000","last_undegraded":"2024-04-24T23:14:28.931020+0000","last_fullsized":"2024-04-24T23:14:28.931020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:00.837670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928366+0000","last_change":"2024-04-24T23:12:28.002875+0000","last_active":"2024-04-24T23:14:28.928366+0000","last_peered":"2024-04-24T23:14:28.928366+0000","last_clean":"2024-04-24T23:14:28.928366+0000","last_became_active":"2024-04-24T23:12:28.002572+0000","last_became_peered":"2024-04-24T23:12:28.002572+0000","last_unstale":"2024-04-24T23:14:28.928366+0000","last_undegraded":"2024-04-24T23:14:28.928366+0000","last_fullsized":"2024-04-24T23:14:28.928366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:55.931371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296043+0000","last_change":"2024-04-24T23:12:32.024729+0000","last_active":"2024-04-24T23:14:29.296043+0000","last_peered":"2024-04-24T23:14:29.296043+0000","last_clean":"2024-04-24T23:14:29.296043+0000","last_became_active":"2024-04-24T23:12:32.024316+0000","last_became_peered":"2024-04-24T23:12:32.024316+0000","last_unstale":"2024-04-24T23:14:29.296043+0000","last_undegraded":"2024-04-24T23:14:29.296043+0000","last_fullsized":"2024-04-24T23:14:29.296043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:21.668782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012232+0000","last_change":"2024-04-24T23:12:27.003435+0000","last_active":"2024-04-24T23:14:29.012232+0000","last_peered":"2024-04-24T23:14:29.012232+0000","last_clean":"2024-04-24T23:14:29.012232+0000","last_became_active":"2024-04-24T23:12:27.003101+0000","last_became_peered":"2024-04-24T23:12:27.003101+0000","last_unstale":"2024-04-24T23:14:29.012232+0000","last_undegraded":"2024-04-24T23:14:29.012232+0000","last_fullsized":"2024-04-24T23:14:29.012232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:42.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931602+0000","last_change":"2024-04-24T23:12:28.005056+0000","last_active":"2024-04-24T23:14:28.931602+0000","last_peered":"2024-04-24T23:14:28.931602+0000","last_clean":"2024-04-24T23:14:28.931602+0000","last_became_active":"2024-04-24T23:12:28.004725+0000","last_became_peered":"2024-04-24T23:12:28.004725+0000","last_unstale":"2024-04-24T23:14:28.931602+0000","last_undegraded":"2024-04-24T23:14:28.931602+0000","last_fullsized":"2024-04-24T23:14:28.931602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:49.384950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928478+0000","last_change":"2024-04-24T23:12:32.032947+0000","last_active":"2024-04-24T23:14:28.928478+0000","last_peered":"2024-04-24T23:14:28.928478+0000","last_clean":"2024-04-24T23:14:28.928478+0000","last_became_active":"2024-04-24T23:12:32.032560+0000","last_became_peered":"2024-04-24T23:12:32.032560+0000","last_unstale":"2024-04-24T23:14:28.928478+0000","last_undegraded":"2024-04-24T23:14:28.928478+0000","last_fullsized":"2024-04-24T23:14:28.928478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:29.282938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":265,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.034588+0000","last_change":"2024-04-24T23:12:27.000011+0000","last_active":"2024-04-24T23:14:48.034588+0000","last_peered":"2024-04-24T23:14:48.034588+0000","last_clean":"2024-04-24T23:14:48.034588+0000","last_became_active":"2024-04-24T23:12:26.999572+0000","last_became_peered":"2024-04-24T23:12:26.999572+0000","last_unstale":"2024-04-24T23:14:48.034588+0000","last_undegraded":"2024-04-24T23:14:48.034588+0000","last_fullsized":"2024-04-24T23:14:48.034588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:50.453964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515375+0000","last_change":"2024-04-24T23:12:28.012808+0000","last_active":"2024-04-24T23:14:29.515375+0000","last_peered":"2024-04-24T23:14:29.515375+0000","last_clean":"2024-04-24T23:14:29.515375+0000","last_became_active":"2024-04-24T23:12:28.012406+0000","last_became_peered":"2024-04-24T23:12:28.012406+0000","last_unstale":"2024-04-24T23:14:29.515375+0000","last_undegraded":"2024-04-24T23:14:29.515375+0000","last_fullsized":"2024-04-24T23:14:29.515375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:19.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931289+0000","last_change":"2024-04-24T23:12:32.027156+0000","last_active":"2024-04-24T23:14:28.931289+0000","last_peered":"2024-04-24T23:14:28.931289+0000","last_clean":"2024-04-24T23:14:28.931289+0000","last_became_active":"2024-04-24T23:12:32.026743+0000","last_became_peered":"2024-04-24T23:12:32.026743+0000","last_unstale":"2024-04-24T23:14:28.931289+0000","last_undegraded":"2024-04-24T23:14:28.931289+0000","last_fullsized":"2024-04-24T23:14:28.931289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:43.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009591+0000","last_change":"2024-04-24T23:12:27.001832+0000","last_active":"2024-04-24T23:14:29.009591+0000","last_peered":"2024-04-24T23:14:29.009591+0000","last_clean":"2024-04-24T23:14:29.009591+0000","last_became_active":"2024-04-24T23:12:27.001482+0000","last_became_peered":"2024-04-24T23:12:27.001482+0000","last_unstale":"2024-04-24T23:14:29.009591+0000","last_undegraded":"2024-04-24T23:14:29.009591+0000","last_fullsized":"2024-04-24T23:14:29.009591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:28.949767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296224+0000","last_change":"2024-04-24T23:13:27.960551+0000","last_active":"2024-04-24T23:14:29.296224+0000","last_peered":"2024-04-24T23:14:29.296224+0000","last_clean":"2024-04-24T23:14:29.296224+0000","last_became_active":"2024-04-24T23:13:27.960264+0000","last_became_peered":"2024-04-24T23:13:27.960264+0000","last_unstale":"2024-04-24T23:14:29.296224+0000","last_undegraded":"2024-04-24T23:14:29.296224+0000","last_fullsized":"2024-04-24T23:14:29.296224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:38.758688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008555+0000","last_change":"2024-04-24T23:12:32.023007+0000","last_active":"2024-04-24T23:14:29.008555+0000","last_peered":"2024-04-24T23:14:29.008555+0000","last_clean":"2024-04-24T23:14:29.008555+0000","last_became_active":"2024-04-24T23:12:32.021324+0000","last_became_peered":"2024-04-24T23:12:32.021324+0000","last_unstale":"2024-04-24T23:14:29.008555+0000","last_undegraded":"2024-04-24T23:14:29.008555+0000","last_fullsized":"2024-04-24T23:14:29.008555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:41.542778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012520+0000","last_change":"2024-04-24T23:12:26.990562+0000","last_active":"2024-04-24T23:14:29.012520+0000","last_peered":"2024-04-24T23:14:29.012520+0000","last_clean":"2024-04-24T23:14:29.012520+0000","last_became_active":"2024-04-24T23:12:26.990146+0000","last_became_peered":"2024-04-24T23:12:26.990146+0000","last_unstale":"2024-04-24T23:14:29.012520+0000","last_undegraded":"2024-04-24T23:14:29.012520+0000","last_fullsized":"2024-04-24T23:14:29.012520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.621051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515294+0000","last_change":"2024-04-24T23:12:28.000954+0000","last_active":"2024-04-24T23:14:29.515294+0000","last_peered":"2024-04-24T23:14:29.515294+0000","last_clean":"2024-04-24T23:14:29.515294+0000","last_became_active":"2024-04-24T23:12:28.000610+0000","last_became_peered":"2024-04-24T23:12:28.000610+0000","last_unstale":"2024-04-24T23:14:29.515294+0000","last_undegraded":"2024-04-24T23:14:29.515294+0000","last_fullsized":"2024-04-24T23:14:29.515294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:32.585919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.516500+0000","last_change":"2024-04-24T23:14:29.516500+0000","last_active":"2024-04-24T23:14:29.516500+0000","last_peered":"2024-04-24T23:14:29.516500+0000","last_clean":"2024-04-24T23:14:29.516500+0000","last_became_active":"2024-04-24T23:14:29.516159+0000","last_became_peered":"2024-04-24T23:14:29.516159+0000","last_unstale":"2024-04-24T23:14:29.516500+0000","last_undegraded":"2024-04-24T23:14:29.516500+0000","last_fullsized":"2024-04-24T23:14:29.516500+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:11.290978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":312,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.043073+0000","last_change":"2024-04-24T23:12:26.996719+0000","last_active":"2024-04-24T23:14:48.043073+0000","last_peered":"2024-04-24T23:14:48.043073+0000","last_clean":"2024-04-24T23:14:48.043073+0000","last_became_active":"2024-04-24T23:12:26.996360+0000","last_became_peered":"2024-04-24T23:12:26.996360+0000","last_unstale":"2024-04-24T23:14:48.043073+0000","last_undegraded":"2024-04-24T23:14:48.043073+0000","last_fullsized":"2024-04-24T23:14:48.043073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.526902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63913,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":183,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932164+0000","last_change":"2024-04-24T23:12:28.002889+0000","last_active":"2024-04-24T23:14:28.932164+0000","last_peered":"2024-04-24T23:14:28.932164+0000","last_clean":"2024-04-24T23:14:28.932164+0000","last_became_active":"2024-04-24T23:12:28.002540+0000","last_became_peered":"2024-04-24T23:12:28.002540+0000","last_unstale":"2024-04-24T23:14:28.932164+0000","last_undegraded":"2024-04-24T23:14:28.932164+0000","last_fullsized":"2024-04-24T23:14:28.932164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:57.792628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012875+0000","last_change":"2024-04-24T23:12:32.031160+0000","last_active":"2024-04-24T23:14:29.012875+0000","last_peered":"2024-04-24T23:14:29.012875+0000","last_clean":"2024-04-24T23:14:29.012875+0000","last_became_active":"2024-04-24T23:12:32.030719+0000","last_became_peered":"2024-04-24T23:12:32.030719+0000","last_unstale":"2024-04-24T23:14:29.012875+0000","last_undegraded":"2024-04-24T23:14:29.012875+0000","last_fullsized":"2024-04-24T23:14:29.012875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:45.648011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515047+0000","last_change":"2024-04-24T23:13:27.967294+0000","last_active":"2024-04-24T23:14:29.515047+0000","last_peered":"2024-04-24T23:14:29.515047+0000","last_clean":"2024-04-24T23:14:29.515047+0000","last_became_active":"2024-04-24T23:13:27.958064+0000","last_became_peered":"2024-04-24T23:13:27.958064+0000","last_unstale":"2024-04-24T23:14:29.515047+0000","last_undegraded":"2024-04-24T23:14:29.515047+0000","last_fullsized":"2024-04-24T23:14:29.515047+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:08.824502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931284+0000","last_change":"2024-04-24T23:12:28.003211+0000","last_active":"2024-04-24T23:14:28.931284+0000","last_peered":"2024-04-24T23:14:28.931284+0000","last_clean":"2024-04-24T23:14:28.931284+0000","last_became_active":"2024-04-24T23:12:28.002897+0000","last_became_peered":"2024-04-24T23:12:28.002897+0000","last_unstale":"2024-04-24T23:14:28.931284+0000","last_undegraded":"2024-04-24T23:14:28.931284+0000","last_fullsized":"2024-04-24T23:14:28.931284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.187239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515101+0000","last_change":"2024-04-24T23:12:32.028857+0000","last_active":"2024-04-24T23:14:29.515101+0000","last_peered":"2024-04-24T23:14:29.515101+0000","last_clean":"2024-04-24T23:14:29.515101+0000","last_became_active":"2024-04-24T23:12:32.028455+0000","last_became_peered":"2024-04-24T23:12:32.028455+0000","last_unstale":"2024-04-24T23:14:29.515101+0000","last_undegraded":"2024-04-24T23:14:29.515101+0000","last_fullsized":"2024-04-24T23:14:29.515101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:23.073895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012600+0000","last_change":"2024-04-24T23:12:27.003525+0000","last_active":"2024-04-24T23:14:29.012600+0000","last_peered":"2024-04-24T23:14:29.012600+0000","last_clean":"2024-04-24T23:14:29.012600+0000","last_became_active":"2024-04-24T23:12:27.003203+0000","last_became_peered":"2024-04-24T23:12:27.003203+0000","last_unstale":"2024-04-24T23:14:29.012600+0000","last_undegraded":"2024-04-24T23:14:29.012600+0000","last_fullsized":"2024-04-24T23:14:29.012600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:18.521925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009356+0000","last_change":"2024-04-24T23:12:28.017111+0000","last_active":"2024-04-24T23:14:29.009356+0000","last_peered":"2024-04-24T23:14:29.009356+0000","last_clean":"2024-04-24T23:14:29.009356+0000","last_became_active":"2024-04-24T23:12:28.016725+0000","last_became_peered":"2024-04-24T23:12:28.016725+0000","last_unstale":"2024-04-24T23:14:29.009356+0000","last_undegraded":"2024-04-24T23:14:29.009356+0000","last_fullsized":"2024-04-24T23:14:29.009356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:25.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.297405+0000","last_change":"2024-04-24T23:14:29.297405+0000","last_active":"2024-04-24T23:14:29.297405+0000","last_peered":"2024-04-24T23:14:29.297405+0000","last_clean":"2024-04-24T23:14:29.297405+0000","last_became_active":"2024-04-24T23:14:29.296936+0000","last_became_peered":"2024-04-24T23:14:29.296936+0000","last_unstale":"2024-04-24T23:14:29.297405+0000","last_undegraded":"2024-04-24T23:14:29.297405+0000","last_fullsized":"2024-04-24T23:14:29.297405+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:53.458679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:50.035879+0000","last_change":"2024-04-24T23:12:27.002797+0000","last_active":"2024-04-24T23:14:50.035879+0000","last_peered":"2024-04-24T23:14:50.035879+0000","last_clean":"2024-04-24T23:14:50.035879+0000","last_became_active":"2024-04-24T23:12:27.002086+0000","last_became_peered":"2024-04-24T23:12:27.002086+0000","last_unstale":"2024-04-24T23:14:50.035879+0000","last_undegraded":"2024-04-24T23:14:50.035879+0000","last_fullsized":"2024-04-24T23:14:50.035879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:55.025453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009610+0000","last_change":"2024-04-24T23:13:27.955130+0000","last_active":"2024-04-24T23:14:29.009610+0000","last_peered":"2024-04-24T23:14:29.009610+0000","last_clean":"2024-04-24T23:14:29.009610+0000","last_became_active":"2024-04-24T23:13:27.954784+0000","last_became_peered":"2024-04-24T23:13:27.954784+0000","last_unstale":"2024-04-24T23:14:29.009610+0000","last_undegraded":"2024-04-24T23:14:29.009610+0000","last_fullsized":"2024-04-24T23:14:29.009610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:10.574771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931931+0000","last_change":"2024-04-24T23:12:32.029104+0000","last_active":"2024-04-24T23:14:28.931931+0000","last_peered":"2024-04-24T23:14:28.931931+0000","last_clean":"2024-04-24T23:14:28.931931+0000","last_became_active":"2024-04-24T23:12:32.028704+0000","last_became_peered":"2024-04-24T23:12:32.028704+0000","last_unstale":"2024-04-24T23:14:28.931931+0000","last_undegraded":"2024-04-24T23:14:28.931931+0000","last_fullsized":"2024-04-24T23:14:28.931931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:33.200884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.514991+0000","last_change":"2024-04-24T23:13:27.976835+0000","last_active":"2024-04-24T23:14:29.514991+0000","last_peered":"2024-04-24T23:14:29.514991+0000","last_clean":"2024-04-24T23:14:29.514991+0000","last_became_active":"2024-04-24T23:13:27.959684+0000","last_became_peered":"2024-04-24T23:13:27.959684+0000","last_unstale":"2024-04-24T23:14:29.514991+0000","last_undegraded":"2024-04-24T23:14:29.514991+0000","last_fullsized":"2024-04-24T23:14:29.514991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:24.327964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012405+0000","last_change":"2024-04-24T23:12:28.010510+0000","last_active":"2024-04-24T23:14:29.012405+0000","last_peered":"2024-04-24T23:14:29.012405+0000","last_clean":"2024-04-24T23:14:29.012405+0000","last_became_active":"2024-04-24T23:12:28.009244+0000","last_became_peered":"2024-04-24T23:12:28.009244+0000","last_unstale":"2024-04-24T23:14:29.012405+0000","last_undegraded":"2024-04-24T23:14:29.012405+0000","last_fullsized":"2024-04-24T23:14:29.012405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.146563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012453+0000","last_change":"2024-04-24T23:12:32.031156+0000","last_active":"2024-04-24T23:14:29.012453+0000","last_peered":"2024-04-24T23:14:29.012453+0000","last_clean":"2024-04-24T23:14:29.012453+0000","last_became_active":"2024-04-24T23:12:32.030653+0000","last_became_peered":"2024-04-24T23:12:32.030653+0000","last_unstale":"2024-04-24T23:14:29.012453+0000","last_undegraded":"2024-04-24T23:14:29.012453+0000","last_fullsized":"2024-04-24T23:14:29.012453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:48.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932509+0000","last_change":"2024-04-24T23:12:26.998003+0000","last_active":"2024-04-24T23:14:28.932509+0000","last_peered":"2024-04-24T23:14:28.932509+0000","last_clean":"2024-04-24T23:14:28.932509+0000","last_became_active":"2024-04-24T23:12:26.997661+0000","last_became_peered":"2024-04-24T23:12:26.997661+0000","last_unstale":"2024-04-24T23:14:28.932509+0000","last_undegraded":"2024-04-24T23:14:28.932509+0000","last_fullsized":"2024-04-24T23:14:28.932509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:17.995397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515371+0000","last_change":"2024-04-24T23:12:28.004822+0000","last_active":"2024-04-24T23:14:29.515371+0000","last_peered":"2024-04-24T23:14:29.515371+0000","last_clean":"2024-04-24T23:14:29.515371+0000","last_became_active":"2024-04-24T23:12:28.004334+0000","last_became_peered":"2024-04-24T23:12:28.004334+0000","last_unstale":"2024-04-24T23:14:29.515371+0000","last_undegraded":"2024-04-24T23:14:29.515371+0000","last_fullsized":"2024-04-24T23:14:29.515371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.717796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931518+0000","last_change":"2024-04-24T23:12:32.085309+0000","last_active":"2024-04-24T23:14:28.931518+0000","last_peered":"2024-04-24T23:14:28.931518+0000","last_clean":"2024-04-24T23:14:28.931518+0000","last_became_active":"2024-04-24T23:12:32.084892+0000","last_became_peered":"2024-04-24T23:12:32.084892+0000","last_unstale":"2024-04-24T23:14:28.931518+0000","last_undegraded":"2024-04-24T23:14:28.931518+0000","last_fullsized":"2024-04-24T23:14:28.931518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:20.318901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932253+0000","last_change":"2024-04-24T23:12:27.005096+0000","last_active":"2024-04-24T23:14:28.932253+0000","last_peered":"2024-04-24T23:14:28.932253+0000","last_clean":"2024-04-24T23:14:28.932253+0000","last_became_active":"2024-04-24T23:12:27.004699+0000","last_became_peered":"2024-04-24T23:12:27.004699+0000","last_unstale":"2024-04-24T23:14:28.932253+0000","last_undegraded":"2024-04-24T23:14:28.932253+0000","last_fullsized":"2024-04-24T23:14:28.932253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:58.225361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931600+0000","last_change":"2024-04-24T23:12:28.012230+0000","last_active":"2024-04-24T23:14:28.931600+0000","last_peered":"2024-04-24T23:14:28.931600+0000","last_clean":"2024-04-24T23:14:28.931600+0000","last_became_active":"2024-04-24T23:12:28.011900+0000","last_became_peered":"2024-04-24T23:12:28.011900+0000","last_unstale":"2024-04-24T23:14:28.931600+0000","last_undegraded":"2024-04-24T23:14:28.931600+0000","last_fullsized":"2024-04-24T23:14:28.931600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.780548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932162+0000","last_change":"2024-04-24T23:12:32.026388+0000","last_active":"2024-04-24T23:14:28.932162+0000","last_peered":"2024-04-24T23:14:28.932162+0000","last_clean":"2024-04-24T23:14:28.932162+0000","last_became_active":"2024-04-24T23:12:32.025994+0000","last_became_peered":"2024-04-24T23:12:32.025994+0000","last_unstale":"2024-04-24T23:14:28.932162+0000","last_undegraded":"2024-04-24T23:14:28.932162+0000","last_fullsized":"2024-04-24T23:14:28.932162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:13.062781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":153,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.034564+0000","last_change":"2024-04-24T23:12:27.005080+0000","last_active":"2024-04-24T23:14:49.034564+0000","last_peered":"2024-04-24T23:14:49.034564+0000","last_clean":"2024-04-24T23:14:49.034564+0000","last_became_active":"2024-04-24T23:12:27.004740+0000","last_became_peered":"2024-04-24T23:12:27.004740+0000","last_unstale":"2024-04-24T23:14:49.034564+0000","last_undegraded":"2024-04-24T23:14:49.034564+0000","last_fullsized":"2024-04-24T23:14:49.034564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:22.502399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931619+0000","last_change":"2024-04-24T23:12:28.008016+0000","last_active":"2024-04-24T23:14:28.931619+0000","last_peered":"2024-04-24T23:14:28.931619+0000","last_clean":"2024-04-24T23:14:28.931619+0000","last_became_active":"2024-04-24T23:12:28.007139+0000","last_became_peered":"2024-04-24T23:12:28.007139+0000","last_unstale":"2024-04-24T23:14:28.931619+0000","last_undegraded":"2024-04-24T23:14:28.931619+0000","last_fullsized":"2024-04-24T23:14:28.931619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.698193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932114+0000","last_change":"2024-04-24T23:12:32.085290+0000","last_active":"2024-04-24T23:14:28.932114+0000","last_peered":"2024-04-24T23:14:28.932114+0000","last_clean":"2024-04-24T23:14:28.932114+0000","last_became_active":"2024-04-24T23:12:32.084852+0000","last_became_peered":"2024-04-24T23:12:32.084852+0000","last_unstale":"2024-04-24T23:14:28.932114+0000","last_undegraded":"2024-04-24T23:14:28.932114+0000","last_fullsized":"2024-04-24T23:14:28.932114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.753633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012753+0000","last_change":"2024-04-24T23:12:28.007552+0000","last_active":"2024-04-24T23:14:29.012753+0000","last_peered":"2024-04-24T23:14:29.012753+0000","last_clean":"2024-04-24T23:14:29.012753+0000","last_became_active":"2024-04-24T23:12:28.007196+0000","last_became_peered":"2024-04-24T23:12:28.007196+0000","last_unstale":"2024-04-24T23:14:29.012753+0000","last_undegraded":"2024-04-24T23:14:29.012753+0000","last_fullsized":"2024-04-24T23:14:29.012753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:15.695485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.032904+0000","last_change":"2024-04-24T23:12:26.990814+0000","last_active":"2024-04-24T23:14:48.032904+0000","last_peered":"2024-04-24T23:14:48.032904+0000","last_clean":"2024-04-24T23:14:48.032904+0000","last_became_active":"2024-04-24T23:12:26.990476+0000","last_became_peered":"2024-04-24T23:12:26.990476+0000","last_unstale":"2024-04-24T23:14:48.032904+0000","last_undegraded":"2024-04-24T23:14:48.032904+0000","last_fullsized":"2024-04-24T23:14:48.032904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:32.645770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009649+0000","last_change":"2024-04-24T23:12:32.022039+0000","last_active":"2024-04-24T23:14:29.009649+0000","last_peered":"2024-04-24T23:14:29.009649+0000","last_clean":"2024-04-24T23:14:29.009649+0000","last_became_active":"2024-04-24T23:12:32.021514+0000","last_became_peered":"2024-04-24T23:12:32.021514+0000","last_unstale":"2024-04-24T23:14:29.009649+0000","last_undegraded":"2024-04-24T23:14:29.009649+0000","last_fullsized":"2024-04-24T23:14:29.009649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:42.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928756+0000","last_change":"2024-04-24T23:12:28.008676+0000","last_active":"2024-04-24T23:14:28.928756+0000","last_peered":"2024-04-24T23:14:28.928756+0000","last_clean":"2024-04-24T23:14:28.928756+0000","last_became_active":"2024-04-24T23:12:28.008376+0000","last_became_peered":"2024-04-24T23:12:28.008376+0000","last_unstale":"2024-04-24T23:14:28.928756+0000","last_undegraded":"2024-04-24T23:14:28.928756+0000","last_fullsized":"2024-04-24T23:14:28.928756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:15.358332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008483+0000","last_change":"2024-04-24T23:12:26.994952+0000","last_active":"2024-04-24T23:14:29.008483+0000","last_peered":"2024-04-24T23:14:29.008483+0000","last_clean":"2024-04-24T23:14:29.008483+0000","last_became_active":"2024-04-24T23:12:26.994556+0000","last_became_peered":"2024-04-24T23:12:26.994556+0000","last_unstale":"2024-04-24T23:14:29.008483+0000","last_undegraded":"2024-04-24T23:14:29.008483+0000","last_fullsized":"2024-04-24T23:14:29.008483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.257427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009006+0000","last_change":"2024-04-24T23:11:16.721876+0000","last_active":"2024-04-24T23:14:29.009006+0000","last_peered":"2024-04-24T23:14:29.009006+0000","last_clean":"2024-04-24T23:14:29.009006+0000","last_became_active":"2024-04-24T23:11:16.721291+0000","last_became_peered":"2024-04-24T23:11:16.721291+0000","last_unstale":"2024-04-24T23:14:29.009006+0000","last_undegraded":"2024-04-24T23:14:29.009006+0000","last_fullsized":"2024-04-24T23:14:29.009006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_clean_scrub_stamp":"2024-04-24T23:07:51.257753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.458939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.013052+0000","last_change":"2024-04-24T23:12:32.085905+0000","last_active":"2024-04-24T23:14:29.013052+0000","last_peered":"2024-04-24T23:14:29.013052+0000","last_clean":"2024-04-24T23:14:29.013052+0000","last_became_active":"2024-04-24T23:12:32.085374+0000","last_became_peered":"2024-04-24T23:12:32.085374+0000","last_unstale":"2024-04-24T23:14:29.013052+0000","last_undegraded":"2024-04-24T23:14:29.013052+0000","last_fullsized":"2024-04-24T23:14:29.013052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:16.290651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931162+0000","last_change":"2024-04-24T23:12:27.999880+0000","last_active":"2024-04-24T23:14:28.931162+0000","last_peered":"2024-04-24T23:14:28.931162+0000","last_clean":"2024-04-24T23:14:28.931162+0000","last_became_active":"2024-04-24T23:12:27.999543+0000","last_became_peered":"2024-04-24T23:12:27.999543+0000","last_unstale":"2024-04-24T23:14:28.931162+0000","last_undegraded":"2024-04-24T23:14:28.931162+0000","last_fullsized":"2024-04-24T23:14:28.931162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:44.758968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009273+0000","last_change":"2024-04-24T23:13:28.341009+0000","last_active":"2024-04-24T23:14:29.009273+0000","last_peered":"2024-04-24T23:14:29.009273+0000","last_clean":"2024-04-24T23:14:29.009273+0000","last_became_active":"2024-04-24T23:13:28.338192+0000","last_became_peered":"2024-04-24T23:13:28.338192+0000","last_unstale":"2024-04-24T23:14:29.009273+0000","last_undegraded":"2024-04-24T23:14:29.009273+0000","last_fullsized":"2024-04-24T23:14:29.009273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:11.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515225+0000","last_change":"2024-04-24T23:12:32.024952+0000","last_active":"2024-04-24T23:14:29.515225+0000","last_peered":"2024-04-24T23:14:29.515225+0000","last_clean":"2024-04-24T23:14:29.515225+0000","last_became_active":"2024-04-24T23:12:32.024404+0000","last_became_peered":"2024-04-24T23:12:32.024404+0000","last_unstale":"2024-04-24T23:14:29.515225+0000","last_undegraded":"2024-04-24T23:14:29.515225+0000","last_fullsized":"2024-04-24T23:14:29.515225+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.377794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932193+0000","last_change":"2024-04-24T23:12:27.007191+0000","last_active":"2024-04-24T23:14:28.932193+0000","last_peered":"2024-04-24T23:14:28.932193+0000","last_clean":"2024-04-24T23:14:28.932193+0000","last_became_active":"2024-04-24T23:12:27.006867+0000","last_became_peered":"2024-04-24T23:12:27.006867+0000","last_unstale":"2024-04-24T23:14:28.932193+0000","last_undegraded":"2024-04-24T23:14:28.932193+0000","last_fullsized":"2024-04-24T23:14:28.932193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:58.112293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932401+0000","last_change":"2024-04-24T23:12:27.996322+0000","last_active":"2024-04-24T23:14:28.932401+0000","last_peered":"2024-04-24T23:14:28.932401+0000","last_clean":"2024-04-24T23:14:28.932401+0000","last_became_active":"2024-04-24T23:12:27.995991+0000","last_became_peered":"2024-04-24T23:12:27.995991+0000","last_unstale":"2024-04-24T23:14:28.932401+0000","last_undegraded":"2024-04-24T23:14:28.932401+0000","last_fullsized":"2024-04-24T23:14:28.932401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:08.960347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928414+0000","last_change":"2024-04-24T23:12:32.084959+0000","last_active":"2024-04-24T23:14:28.928414+0000","last_peered":"2024-04-24T23:14:28.928414+0000","last_clean":"2024-04-24T23:14:28.928414+0000","last_became_active":"2024-04-24T23:12:32.084553+0000","last_became_peered":"2024-04-24T23:12:32.084553+0000","last_unstale":"2024-04-24T23:14:28.928414+0000","last_undegraded":"2024-04-24T23:14:28.928414+0000","last_fullsized":"2024-04-24T23:14:28.928414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.836337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932585+0000","last_change":"2024-04-24T23:12:27.007742+0000","last_active":"2024-04-24T23:14:28.932585+0000","last_peered":"2024-04-24T23:14:28.932585+0000","last_clean":"2024-04-24T23:14:28.932585+0000","last_became_active":"2024-04-24T23:12:27.007423+0000","last_became_peered":"2024-04-24T23:12:27.007423+0000","last_unstale":"2024-04-24T23:14:28.932585+0000","last_undegraded":"2024-04-24T23:14:28.932585+0000","last_fullsized":"2024-04-24T23:14:28.932585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:39.478356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928898+0000","last_change":"2024-04-24T23:12:28.010961+0000","last_active":"2024-04-24T23:14:28.928898+0000","last_peered":"2024-04-24T23:14:28.928898+0000","last_clean":"2024-04-24T23:14:28.928898+0000","last_became_active":"2024-04-24T23:12:28.010641+0000","last_became_peered":"2024-04-24T23:12:28.010641+0000","last_unstale":"2024-04-24T23:14:28.928898+0000","last_undegraded":"2024-04-24T23:14:28.928898+0000","last_fullsized":"2024-04-24T23:14:28.928898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:43.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929416+0000","last_change":"2024-04-24T23:12:32.023687+0000","last_active":"2024-04-24T23:14:28.929416+0000","last_peered":"2024-04-24T23:14:28.929416+0000","last_clean":"2024-04-24T23:14:28.929416+0000","last_became_active":"2024-04-24T23:12:32.023052+0000","last_became_peered":"2024-04-24T23:12:32.023052+0000","last_unstale":"2024-04-24T23:14:28.929416+0000","last_undegraded":"2024-04-24T23:14:28.929416+0000","last_fullsized":"2024-04-24T23:14:28.929416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:39.854030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008925+0000","last_change":"2024-04-24T23:12:27.009743+0000","last_active":"2024-04-24T23:14:29.008925+0000","last_peered":"2024-04-24T23:14:29.008925+0000","last_clean":"2024-04-24T23:14:29.008925+0000","last_became_active":"2024-04-24T23:12:27.009343+0000","last_became_peered":"2024-04-24T23:12:27.009343+0000","last_unstale":"2024-04-24T23:14:29.008925+0000","last_undegraded":"2024-04-24T23:14:29.008925+0000","last_fullsized":"2024-04-24T23:14:29.008925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:20.387738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296261+0000","last_change":"2024-04-24T23:13:27.961769+0000","last_active":"2024-04-24T23:14:29.296261+0000","last_peered":"2024-04-24T23:14:29.296261+0000","last_clean":"2024-04-24T23:14:29.296261+0000","last_became_active":"2024-04-24T23:13:27.961460+0000","last_became_peered":"2024-04-24T23:13:27.961460+0000","last_unstale":"2024-04-24T23:14:29.296261+0000","last_undegraded":"2024-04-24T23:14:29.296261+0000","last_fullsized":"2024-04-24T23:14:29.296261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:01.012688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009238+0000","last_change":"2024-04-24T23:12:32.086209+0000","last_active":"2024-04-24T23:14:29.009238+0000","last_peered":"2024-04-24T23:14:29.009238+0000","last_clean":"2024-04-24T23:14:29.009238+0000","last_became_active":"2024-04-24T23:12:32.085752+0000","last_became_peered":"2024-04-24T23:12:32.085752+0000","last_unstale":"2024-04-24T23:14:29.009238+0000","last_undegraded":"2024-04-24T23:14:29.009238+0000","last_fullsized":"2024-04-24T23:14:29.009238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:52.120905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515056+0000","last_change":"2024-04-24T23:12:26.996355+0000","last_active":"2024-04-24T23:14:29.515056+0000","last_peered":"2024-04-24T23:14:29.515056+0000","last_clean":"2024-04-24T23:14:29.515056+0000","last_became_active":"2024-04-24T23:12:26.995486+0000","last_became_peered":"2024-04-24T23:12:26.995486+0000","last_unstale":"2024-04-24T23:14:29.515056+0000","last_undegraded":"2024-04-24T23:14:29.515056+0000","last_fullsized":"2024-04-24T23:14:29.515056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:48.981985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931625+0000","last_change":"2024-04-24T23:12:28.004610+0000","last_active":"2024-04-24T23:14:28.931625+0000","last_peered":"2024-04-24T23:14:28.931625+0000","last_clean":"2024-04-24T23:14:28.931625+0000","last_became_active":"2024-04-24T23:12:28.004051+0000","last_became_peered":"2024-04-24T23:12:28.004051+0000","last_unstale":"2024-04-24T23:14:28.931625+0000","last_undegraded":"2024-04-24T23:14:28.931625+0000","last_fullsized":"2024-04-24T23:14:28.931625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:42.656954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.014680+0000","last_change":"2024-04-24T23:14:29.014103+0000","last_active":"2024-04-24T23:14:29.014680+0000","last_peered":"2024-04-24T23:14:29.014680+0000","last_clean":"2024-04-24T23:14:29.014680+0000","last_became_active":"2024-04-24T23:14:29.013472+0000","last_became_peered":"2024-04-24T23:14:29.013472+0000","last_unstale":"2024-04-24T23:14:29.014680+0000","last_undegraded":"2024-04-24T23:14:29.014680+0000","last_fullsized":"2024-04-24T23:14:29.014680+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:37.823011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928576+0000","last_change":"2024-04-24T23:12:27.006357+0000","last_active":"2024-04-24T23:14:28.928576+0000","last_peered":"2024-04-24T23:14:28.928576+0000","last_clean":"2024-04-24T23:14:28.928576+0000","last_became_active":"2024-04-24T23:12:27.006040+0000","last_became_peered":"2024-04-24T23:12:27.006040+0000","last_unstale":"2024-04-24T23:14:28.928576+0000","last_undegraded":"2024-04-24T23:14:28.928576+0000","last_fullsized":"2024-04-24T23:14:28.928576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:27.520663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515601+0000","last_change":"2024-04-24T23:12:28.008909+0000","last_active":"2024-04-24T23:14:29.515601+0000","last_peered":"2024-04-24T23:14:29.515601+0000","last_clean":"2024-04-24T23:14:29.515601+0000","last_became_active":"2024-04-24T23:12:28.008552+0000","last_became_peered":"2024-04-24T23:12:28.008552+0000","last_unstale":"2024-04-24T23:14:29.515601+0000","last_undegraded":"2024-04-24T23:14:29.515601+0000","last_fullsized":"2024-04-24T23:14:29.515601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:20.628234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008834+0000","last_change":"2024-04-24T23:12:32.029720+0000","last_active":"2024-04-24T23:14:29.008834+0000","last_peered":"2024-04-24T23:14:29.008834+0000","last_clean":"2024-04-24T23:14:29.008834+0000","last_became_active":"2024-04-24T23:12:32.029311+0000","last_became_peered":"2024-04-24T23:12:32.029311+0000","last_unstale":"2024-04-24T23:14:29.008834+0000","last_undegraded":"2024-04-24T23:14:29.008834+0000","last_fullsized":"2024-04-24T23:14:29.008834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:37.832102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:40.286863+0000","last_change":"2024-04-24T23:12:26.995552+0000","last_active":"2024-04-24T23:14:40.286863+0000","last_peered":"2024-04-24T23:14:40.286863+0000","last_clean":"2024-04-24T23:14:40.286863+0000","last_became_active":"2024-04-24T23:12:26.994922+0000","last_became_peered":"2024-04-24T23:12:26.994922+0000","last_unstale":"2024-04-24T23:14:40.286863+0000","last_undegraded":"2024-04-24T23:14:40.286863+0000","last_fullsized":"2024-04-24T23:14:40.286863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:17.981344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928695+0000","last_change":"2024-04-24T23:12:28.008802+0000","last_active":"2024-04-24T23:14:28.928695+0000","last_peered":"2024-04-24T23:14:28.928695+0000","last_clean":"2024-04-24T23:14:28.928695+0000","last_became_active":"2024-04-24T23:12:28.008496+0000","last_became_peered":"2024-04-24T23:12:28.008496+0000","last_unstale":"2024-04-24T23:14:28.928695+0000","last_undegraded":"2024-04-24T23:14:28.928695+0000","last_fullsized":"2024-04-24T23:14:28.928695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:54.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008767+0000","last_change":"2024-04-24T23:12:32.085555+0000","last_active":"2024-04-24T23:14:29.008767+0000","last_peered":"2024-04-24T23:14:29.008767+0000","last_clean":"2024-04-24T23:14:29.008767+0000","last_became_active":"2024-04-24T23:12:32.085085+0000","last_became_peered":"2024-04-24T23:12:32.085085+0000","last_unstale":"2024-04-24T23:14:29.008767+0000","last_undegraded":"2024-04-24T23:14:29.008767+0000","last_fullsized":"2024-04-24T23:14:29.008767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:24.563964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932112+0000","last_change":"2024-04-24T23:13:28.342962+0000","last_active":"2024-04-24T23:14:28.932112+0000","last_peered":"2024-04-24T23:14:28.932112+0000","last_clean":"2024-04-24T23:14:28.932112+0000","last_became_active":"2024-04-24T23:13:28.339742+0000","last_became_peered":"2024-04-24T23:13:28.339742+0000","last_unstale":"2024-04-24T23:14:28.932112+0000","last_undegraded":"2024-04-24T23:14:28.932112+0000","last_fullsized":"2024-04-24T23:14:28.932112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:36.910863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009444+0000","last_change":"2024-04-24T23:12:28.014865+0000","last_active":"2024-04-24T23:14:29.009444+0000","last_peered":"2024-04-24T23:14:29.009444+0000","last_clean":"2024-04-24T23:14:29.009444+0000","last_became_active":"2024-04-24T23:12:28.014538+0000","last_became_peered":"2024-04-24T23:12:28.014538+0000","last_unstale":"2024-04-24T23:14:29.009444+0000","last_undegraded":"2024-04-24T23:14:29.009444+0000","last_fullsized":"2024-04-24T23:14:29.009444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:55.306883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515116+0000","last_change":"2024-04-24T23:12:32.024849+0000","last_active":"2024-04-24T23:14:29.515116+0000","last_peered":"2024-04-24T23:14:29.515116+0000","last_clean":"2024-04-24T23:14:29.515116+0000","last_became_active":"2024-04-24T23:12:32.024412+0000","last_became_peered":"2024-04-24T23:12:32.024412+0000","last_unstale":"2024-04-24T23:14:29.515116+0000","last_undegraded":"2024-04-24T23:14:29.515116+0000","last_fullsized":"2024-04-24T23:14:29.515116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:16.884565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.036058+0000","last_change":"2024-04-24T23:12:27.007284+0000","last_active":"2024-04-24T23:14:49.036058+0000","last_peered":"2024-04-24T23:14:49.036058+0000","last_clean":"2024-04-24T23:14:49.036058+0000","last_became_active":"2024-04-24T23:12:27.006916+0000","last_became_peered":"2024-04-24T23:12:27.006916+0000","last_unstale":"2024-04-24T23:14:49.036058+0000","last_undegraded":"2024-04-24T23:14:49.036058+0000","last_fullsized":"2024-04-24T23:14:49.036058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:40.333387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.010029+0000","last_change":"2024-04-24T23:12:28.011603+0000","last_active":"2024-04-24T23:14:29.010029+0000","last_peered":"2024-04-24T23:14:29.010029+0000","last_clean":"2024-04-24T23:14:29.010029+0000","last_became_active":"2024-04-24T23:12:28.011268+0000","last_became_peered":"2024-04-24T23:12:28.011268+0000","last_unstale":"2024-04-24T23:14:29.010029+0000","last_undegraded":"2024-04-24T23:14:29.010029+0000","last_fullsized":"2024-04-24T23:14:29.010029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.693024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295803+0000","last_change":"2024-04-24T23:12:32.033965+0000","last_active":"2024-04-24T23:14:29.295803+0000","last_peered":"2024-04-24T23:14:29.295803+0000","last_clean":"2024-04-24T23:14:29.295803+0000","last_became_active":"2024-04-24T23:12:32.033472+0000","last_became_peered":"2024-04-24T23:12:32.033472+0000","last_unstale":"2024-04-24T23:14:29.295803+0000","last_undegraded":"2024-04-24T23:14:29.295803+0000","last_fullsized":"2024-04-24T23:14:29.295803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:05.243047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012764+0000","last_change":"2024-04-24T23:12:26.991981+0000","last_active":"2024-04-24T23:14:29.012764+0000","last_peered":"2024-04-24T23:14:29.012764+0000","last_clean":"2024-04-24T23:14:29.012764+0000","last_became_active":"2024-04-24T23:12:26.990707+0000","last_became_peered":"2024-04-24T23:12:26.990707+0000","last_unstale":"2024-04-24T23:14:29.012764+0000","last_undegraded":"2024-04-24T23:14:29.012764+0000","last_fullsized":"2024-04-24T23:14:29.012764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:36.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009154+0000","last_change":"2024-04-24T23:12:28.010428+0000","last_active":"2024-04-24T23:14:29.009154+0000","last_peered":"2024-04-24T23:14:29.009154+0000","last_clean":"2024-04-24T23:14:29.009154+0000","last_became_active":"2024-04-24T23:12:28.010096+0000","last_became_peered":"2024-04-24T23:12:28.010096+0000","last_unstale":"2024-04-24T23:14:29.009154+0000","last_undegraded":"2024-04-24T23:14:29.009154+0000","last_fullsized":"2024-04-24T23:14:29.009154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:31.949617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012035+0000","last_change":"2024-04-24T23:12:32.023828+0000","last_active":"2024-04-24T23:14:29.012035+0000","last_peered":"2024-04-24T23:14:29.012035+0000","last_clean":"2024-04-24T23:14:29.012035+0000","last_became_active":"2024-04-24T23:12:32.023437+0000","last_became_peered":"2024-04-24T23:12:32.023437+0000","last_unstale":"2024-04-24T23:14:29.012035+0000","last_undegraded":"2024-04-24T23:14:29.012035+0000","last_fullsized":"2024-04-24T23:14:29.012035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.186860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931862+0000","last_change":"2024-04-24T23:12:26.993711+0000","last_active":"2024-04-24T23:14:28.931862+0000","last_peered":"2024-04-24T23:14:28.931862+0000","last_clean":"2024-04-24T23:14:28.931862+0000","last_became_active":"2024-04-24T23:12:26.993368+0000","last_became_peered":"2024-04-24T23:12:26.993368+0000","last_unstale":"2024-04-24T23:14:28.931862+0000","last_undegraded":"2024-04-24T23:14:28.931862+0000","last_fullsized":"2024-04-24T23:14:28.931862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:58.843747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009225+0000","last_change":"2024-04-24T23:12:28.003026+0000","last_active":"2024-04-24T23:14:29.009225+0000","last_peered":"2024-04-24T23:14:29.009225+0000","last_clean":"2024-04-24T23:14:29.009225+0000","last_became_active":"2024-04-24T23:12:28.002693+0000","last_became_peered":"2024-04-24T23:12:28.002693+0000","last_unstale":"2024-04-24T23:14:29.009225+0000","last_undegraded":"2024-04-24T23:14:29.009225+0000","last_fullsized":"2024-04-24T23:14:29.009225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:30.926251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515390+0000","last_change":"2024-04-24T23:12:28.013414+0000","last_active":"2024-04-24T23:14:29.515390+0000","last_peered":"2024-04-24T23:14:29.515390+0000","last_clean":"2024-04-24T23:14:29.515390+0000","last_became_active":"2024-04-24T23:12:28.013030+0000","last_became_peered":"2024-04-24T23:12:28.013030+0000","last_unstale":"2024-04-24T23:14:29.515390+0000","last_undegraded":"2024-04-24T23:14:29.515390+0000","last_fullsized":"2024-04-24T23:14:29.515390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:48.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930086+0000","last_change":"2024-04-24T23:12:28.004821+0000","last_active":"2024-04-24T23:14:28.930086+0000","last_peered":"2024-04-24T23:14:28.930086+0000","last_clean":"2024-04-24T23:14:28.930086+0000","last_became_active":"2024-04-24T23:12:28.003162+0000","last_became_peered":"2024-04-24T23:12:28.003162+0000","last_unstale":"2024-04-24T23:14:28.930086+0000","last_undegraded":"2024-04-24T23:14:28.930086+0000","last_fullsized":"2024-04-24T23:14:28.930086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:59.109665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929419+0000","last_change":"2024-04-24T23:12:28.002115+0000","last_active":"2024-04-24T23:14:28.929419+0000","last_peered":"2024-04-24T23:14:28.929419+0000","last_clean":"2024-04-24T23:14:28.929419+0000","last_became_active":"2024-04-24T23:12:28.001727+0000","last_became_peered":"2024-04-24T23:12:28.001727+0000","last_unstale":"2024-04-24T23:14:28.929419+0000","last_undegraded":"2024-04-24T23:14:28.929419+0000","last_fullsized":"2024-04-24T23:14:28.929419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:14.348843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295994+0000","last_change":"2024-04-24T23:12:28.007784+0000","last_active":"2024-04-24T23:14:29.295994+0000","last_peered":"2024-04-24T23:14:29.295994+0000","last_clean":"2024-04-24T23:14:29.295994+0000","last_became_active":"2024-04-24T23:12:28.007481+0000","last_became_peered":"2024-04-24T23:12:28.007481+0000","last_unstale":"2024-04-24T23:14:29.295994+0000","last_undegraded":"2024-04-24T23:14:29.295994+0000","last_fullsized":"2024-04-24T23:14:29.295994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:33.753448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932358+0000","last_change":"2024-04-24T23:12:28.009100+0000","last_active":"2024-04-24T23:14:28.932358+0000","last_peered":"2024-04-24T23:14:28.932358+0000","last_clean":"2024-04-24T23:14:28.932358+0000","last_became_active":"2024-04-24T23:12:28.008782+0000","last_became_peered":"2024-04-24T23:12:28.008782+0000","last_unstale":"2024-04-24T23:14:28.932358+0000","last_undegraded":"2024-04-24T23:14:28.932358+0000","last_fullsized":"2024-04-24T23:14:28.932358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:44.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515607+0000","last_change":"2024-04-24T23:12:28.008129+0000","last_active":"2024-04-24T23:14:29.515607+0000","last_peered":"2024-04-24T23:14:29.515607+0000","last_clean":"2024-04-24T23:14:29.515607+0000","last_became_active":"2024-04-24T23:12:28.007819+0000","last_became_peered":"2024-04-24T23:12:28.007819+0000","last_unstale":"2024-04-24T23:14:29.515607+0000","last_undegraded":"2024-04-24T23:14:29.515607+0000","last_fullsized":"2024-04-24T23:14:29.515607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.234705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930578+0000","last_change":"2024-04-24T23:12:27.993188+0000","last_active":"2024-04-24T23:14:28.930578+0000","last_peered":"2024-04-24T23:14:28.930578+0000","last_clean":"2024-04-24T23:14:28.930578+0000","last_became_active":"2024-04-24T23:12:27.992832+0000","last_became_peered":"2024-04-24T23:12:27.992832+0000","last_unstale":"2024-04-24T23:14:28.930578+0000","last_undegraded":"2024-04-24T23:14:28.930578+0000","last_fullsized":"2024-04-24T23:14:28.930578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:58.623164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928983+0000","last_change":"2024-04-24T23:12:28.004099+0000","last_active":"2024-04-24T23:14:28.928983+0000","last_peered":"2024-04-24T23:14:28.928983+0000","last_clean":"2024-04-24T23:14:28.928983+0000","last_became_active":"2024-04-24T23:12:28.003802+0000","last_became_peered":"2024-04-24T23:12:28.003802+0000","last_unstale":"2024-04-24T23:14:28.928983+0000","last_undegraded":"2024-04-24T23:14:28.928983+0000","last_fullsized":"2024-04-24T23:14:28.928983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:25.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012613+0000","last_change":"2024-04-24T23:12:28.007700+0000","last_active":"2024-04-24T23:14:29.012613+0000","last_peered":"2024-04-24T23:14:29.012613+0000","last_clean":"2024-04-24T23:14:29.012613+0000","last_became_active":"2024-04-24T23:12:28.002923+0000","last_became_peered":"2024-04-24T23:12:28.002923+0000","last_unstale":"2024-04-24T23:14:29.012613+0000","last_undegraded":"2024-04-24T23:14:29.012613+0000","last_fullsized":"2024-04-24T23:14:29.012613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:46.736096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928635+0000","last_change":"2024-04-24T23:12:28.002194+0000","last_active":"2024-04-24T23:14:28.928635+0000","last_peered":"2024-04-24T23:14:28.928635+0000","last_clean":"2024-04-24T23:14:28.928635+0000","last_became_active":"2024-04-24T23:12:28.001703+0000","last_became_peered":"2024-04-24T23:12:28.001703+0000","last_unstale":"2024-04-24T23:14:28.928635+0000","last_undegraded":"2024-04-24T23:14:28.928635+0000","last_fullsized":"2024-04-24T23:14:28.928635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:20.887025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515261+0000","last_change":"2024-04-24T23:12:28.015980+0000","last_active":"2024-04-24T23:14:29.515261+0000","last_peered":"2024-04-24T23:14:29.515261+0000","last_clean":"2024-04-24T23:14:29.515261+0000","last_became_active":"2024-04-24T23:12:28.013510+0000","last_became_peered":"2024-04-24T23:12:28.013510+0000","last_unstale":"2024-04-24T23:14:29.515261+0000","last_undegraded":"2024-04-24T23:14:29.515261+0000","last_fullsized":"2024-04-24T23:14:29.515261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:37.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931126+0000","last_change":"2024-04-24T23:12:28.001883+0000","last_active":"2024-04-24T23:14:28.931126+0000","last_peered":"2024-04-24T23:14:28.931126+0000","last_clean":"2024-04-24T23:14:28.931126+0000","last_became_active":"2024-04-24T23:12:28.001499+0000","last_became_peered":"2024-04-24T23:12:28.001499+0000","last_unstale":"2024-04-24T23:14:28.931126+0000","last_undegraded":"2024-04-24T23:14:28.931126+0000","last_fullsized":"2024-04-24T23:14:28.931126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:46.805548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515457+0000","last_change":"2024-04-24T23:12:28.004660+0000","last_active":"2024-04-24T23:14:29.515457+0000","last_peered":"2024-04-24T23:14:29.515457+0000","last_clean":"2024-04-24T23:14:29.515457+0000","last_became_active":"2024-04-24T23:12:28.004328+0000","last_became_peered":"2024-04-24T23:12:28.004328+0000","last_unstale":"2024-04-24T23:14:29.515457+0000","last_undegraded":"2024-04-24T23:14:29.515457+0000","last_fullsized":"2024-04-24T23:14:29.515457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:04.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71175,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1192,"num_read_kb":671,"num_write":254,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":213217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":525158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32908,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712244,"statfs":{"total":95995035648,"available":95961337856,"internally_reserved":0,"allocated":1773568,"data_stored":1175707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":529004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037816,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":525174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1122232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":520412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":585047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":521868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":586347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":520120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":524486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1117814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:14:52.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:14:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 -- 172.21.15.112:0/1850126636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70101880 msgr2=0x7f7e70101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 --2- 172.21.15.112:0/1850126636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70101880 0x7f7e70101d00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009ca0 tx=0x7f7e6402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 -- 172.21.15.112:0/1850126636 shutdown_connections 2024-04-24T23:14:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 --2- 172.21.15.112:0/1850126636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e70102240 0x7f7e70108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 --2- 172.21.15.112:0/1850126636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70101880 0x7f7e70101d00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 --2- 172.21.15.112:0/1850126636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e70100690 0x7f7e70100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 -- 172.21.15.112:0/1850126636 >> 172.21.15.112:0/1850126636 conn(0x7f7e700fbc20 msgr2=0x7f7e700fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:52.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 -- 172.21.15.112:0/1850126636 shutdown_connections 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 -- 172.21.15.112:0/1850126636 wait complete. 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.864+0000 7f7e754b3640 1 Processor -- start 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 -- start start 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70100690 0x7f7e701046f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e70101880 0x7f7e70102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e70102240 0x7f7e70103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010c440 con 0x7f7e70100690 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010c2c0 con 0x7f7e70102240 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7010c5c0 con 0x7f7e70101880 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70100690 0x7f7e701046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70100690 0x7f7e701046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39680/0 (socket says 172.21.15.112:39680) 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 -- 172.21.15.112:0/2830208325 learned_addr learned my addr 172.21.15.112:0/2830208325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6e7fc640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e70101880 0x7f7e70102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6f7fe640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e70102240 0x7f7e70103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 -- 172.21.15.112:0/2830208325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e70101880 msgr2=0x7f7e70102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e70101880 0x7f7e70102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 -- 172.21.15.112:0/2830208325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e70102240 msgr2=0x7f7e70103270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e70102240 0x7f7e70103270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 -- 172.21.15.112:0/2830208325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e70103b00 con 0x7f7e70100690 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e6effd640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70100690 0x7f7e701046f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d770 tx=0x7f7e6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e57fff640 1 -- 172.21.15.112:0/2830208325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68017070 con 0x7f7e70100690 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e57fff640 1 -- 172.21.15.112:0/2830208325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7e68009c30 con 0x7f7e70100690 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e701a1c90 con 0x7f7e70100690 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e57fff640 1 -- 172.21.15.112:0/2830208325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68003e40 con 0x7f7e70100690 2024-04-24T23:14:52.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.865+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e701a21b0 con 0x7f7e70100690 2024-04-24T23:14:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.867+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e701a2380 con 0x7f7e70100690 2024-04-24T23:14:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.867+0000 7f7e57fff640 1 -- 172.21.15.112:0/2830208325 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7e68004000 con 0x7f7e70100690 2024-04-24T23:14:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.868+0000 7f7e57fff640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7e580a2b90 0x7f7e580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.868+0000 7f7e57fff640 1 -- 172.21.15.112:0/2830208325 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f7e680ceda0 con 0x7f7e70100690 2024-04-24T23:14:52.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.868+0000 7f7e6e7fc640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7e580a2b90 0x7f7e580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:52.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.868+0000 7f7e6e7fc640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7e580a2b90 0x7f7e580a5050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7e64006010 tx=0x7f7e64009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:52.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:52.872+0000 7f7e57fff640 1 -- 172.21.15.112:0/2830208325 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7e6809b910 con 0x7f7e70100690 2024-04-24T23:14:53.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.048+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e7006a450 con 0x7f7e580a2b90 2024-04-24T23:14:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.057+0000 7f7e57fff640 1 -- 172.21.15.112:0/2830208325 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340539 (secure 0 0 0) 0x7f7e7006a450 con 0x7f7e580a2b90 2024-04-24T23:14:53.062 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:53.064 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-24T23:14:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7e580a2b90 msgr2=0x7f7e580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7e580a2b90 0x7f7e580a5050 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7e64006010 tx=0x7f7e64009360 comp rx=0 tx=0).stop 2024-04-24T23:14:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70100690 msgr2=0x7f7e701046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70100690 0x7f7e701046f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d770 tx=0x7f7e6800dc30 comp rx=0 tx=0).stop 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 shutdown_connections 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7e580a2b90 0x7f7e580a5050 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e70102240 0x7f7e70103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e70101880 0x7f7e70102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 --2- 172.21.15.112:0/2830208325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e70100690 0x7f7e701046f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.060+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 >> 172.21.15.112:0/2830208325 conn(0x7f7e700fbc20 msgr2=0x7f7e70069d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.061+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 shutdown_connections 2024-04-24T23:14:53.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.061+0000 7f7e754b3640 1 -- 172.21.15.112:0/2830208325 wait complete. 2024-04-24T23:14:53.073 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-04-24T23:14:52.822084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661777,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1318,"num_read_kb":780,"num_write":437,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384764,"kb_used_data":14772,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557060,"statfs":{"total":1151940427776,"available":1151546429440,"internally_reserved":0,"allocated":15126528,"data_stored":8257991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003125"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515637+0000","last_change":"2024-04-24T23:12:28.013506+0000","last_active":"2024-04-24T23:14:29.515637+0000","last_peered":"2024-04-24T23:14:29.515637+0000","last_clean":"2024-04-24T23:14:29.515637+0000","last_became_active":"2024-04-24T23:12:28.012652+0000","last_became_peered":"2024-04-24T23:12:28.012652+0000","last_unstale":"2024-04-24T23:14:29.515637+0000","last_undegraded":"2024-04-24T23:14:29.515637+0000","last_fullsized":"2024-04-24T23:14:29.515637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:57.420465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296164+0000","last_change":"2024-04-24T23:12:28.004998+0000","last_active":"2024-04-24T23:14:29.296164+0000","last_peered":"2024-04-24T23:14:29.296164+0000","last_clean":"2024-04-24T23:14:29.296164+0000","last_became_active":"2024-04-24T23:12:28.004706+0000","last_became_peered":"2024-04-24T23:12:28.004706+0000","last_unstale":"2024-04-24T23:14:29.296164+0000","last_undegraded":"2024-04-24T23:14:29.296164+0000","last_fullsized":"2024-04-24T23:14:29.296164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.759515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296026+0000","last_change":"2024-04-24T23:12:28.019170+0000","last_active":"2024-04-24T23:14:29.296026+0000","last_peered":"2024-04-24T23:14:29.296026+0000","last_clean":"2024-04-24T23:14:29.296026+0000","last_became_active":"2024-04-24T23:12:28.018815+0000","last_became_peered":"2024-04-24T23:12:28.018815+0000","last_unstale":"2024-04-24T23:14:29.296026+0000","last_undegraded":"2024-04-24T23:14:29.296026+0000","last_fullsized":"2024-04-24T23:14:29.296026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:43.830598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009162+0000","last_change":"2024-04-24T23:12:28.008662+0000","last_active":"2024-04-24T23:14:29.009162+0000","last_peered":"2024-04-24T23:14:29.009162+0000","last_clean":"2024-04-24T23:14:29.009162+0000","last_became_active":"2024-04-24T23:12:28.008282+0000","last_became_peered":"2024-04-24T23:12:28.008282+0000","last_unstale":"2024-04-24T23:14:29.009162+0000","last_undegraded":"2024-04-24T23:14:29.009162+0000","last_fullsized":"2024-04-24T23:14:29.009162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:17.558899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931843+0000","last_change":"2024-04-24T23:12:28.005212+0000","last_active":"2024-04-24T23:14:28.931843+0000","last_peered":"2024-04-24T23:14:28.931843+0000","last_clean":"2024-04-24T23:14:28.931843+0000","last_became_active":"2024-04-24T23:12:28.004853+0000","last_became_peered":"2024-04-24T23:12:28.004853+0000","last_unstale":"2024-04-24T23:14:28.931843+0000","last_undegraded":"2024-04-24T23:14:28.931843+0000","last_fullsized":"2024-04-24T23:14:28.931843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:33.650922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931438+0000","last_change":"2024-04-24T23:12:28.004546+0000","last_active":"2024-04-24T23:14:28.931438+0000","last_peered":"2024-04-24T23:14:28.931438+0000","last_clean":"2024-04-24T23:14:28.931438+0000","last_became_active":"2024-04-24T23:12:28.004194+0000","last_became_peered":"2024-04-24T23:12:28.004194+0000","last_unstale":"2024-04-24T23:14:28.931438+0000","last_undegraded":"2024-04-24T23:14:28.931438+0000","last_fullsized":"2024-04-24T23:14:28.931438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:57.772271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008625+0000","last_change":"2024-04-24T23:12:28.009436+0000","last_active":"2024-04-24T23:14:29.008625+0000","last_peered":"2024-04-24T23:14:29.008625+0000","last_clean":"2024-04-24T23:14:29.008625+0000","last_became_active":"2024-04-24T23:12:28.008219+0000","last_became_peered":"2024-04-24T23:12:28.008219+0000","last_unstale":"2024-04-24T23:14:29.008625+0000","last_undegraded":"2024-04-24T23:14:29.008625+0000","last_fullsized":"2024-04-24T23:14:29.008625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930928+0000","last_change":"2024-04-24T23:13:27.957345+0000","last_active":"2024-04-24T23:14:28.930928+0000","last_peered":"2024-04-24T23:14:28.930928+0000","last_clean":"2024-04-24T23:14:28.930928+0000","last_became_active":"2024-04-24T23:13:27.956989+0000","last_became_peered":"2024-04-24T23:13:27.956989+0000","last_unstale":"2024-04-24T23:14:28.930928+0000","last_undegraded":"2024-04-24T23:14:28.930928+0000","last_fullsized":"2024-04-24T23:14:28.930928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:42.120100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515468+0000","last_change":"2024-04-24T23:12:28.004948+0000","last_active":"2024-04-24T23:14:29.515468+0000","last_peered":"2024-04-24T23:14:29.515468+0000","last_clean":"2024-04-24T23:14:29.515468+0000","last_became_active":"2024-04-24T23:12:28.004380+0000","last_became_peered":"2024-04-24T23:12:28.004380+0000","last_unstale":"2024-04-24T23:14:29.515468+0000","last_undegraded":"2024-04-24T23:14:29.515468+0000","last_fullsized":"2024-04-24T23:14:29.515468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:53.999210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931753+0000","last_change":"2024-04-24T23:12:28.005192+0000","last_active":"2024-04-24T23:14:28.931753+0000","last_peered":"2024-04-24T23:14:28.931753+0000","last_clean":"2024-04-24T23:14:28.931753+0000","last_became_active":"2024-04-24T23:12:28.004860+0000","last_became_peered":"2024-04-24T23:12:28.004860+0000","last_unstale":"2024-04-24T23:14:28.931753+0000","last_undegraded":"2024-04-24T23:14:28.931753+0000","last_fullsized":"2024-04-24T23:14:28.931753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:21.874408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009736+0000","last_change":"2024-04-24T23:12:28.016098+0000","last_active":"2024-04-24T23:14:29.009736+0000","last_peered":"2024-04-24T23:14:29.009736+0000","last_clean":"2024-04-24T23:14:29.009736+0000","last_became_active":"2024-04-24T23:12:28.015736+0000","last_became_peered":"2024-04-24T23:12:28.015736+0000","last_unstale":"2024-04-24T23:14:29.009736+0000","last_undegraded":"2024-04-24T23:14:29.009736+0000","last_fullsized":"2024-04-24T23:14:29.009736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.071797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009000+0000","last_change":"2024-04-24T23:12:28.010651+0000","last_active":"2024-04-24T23:14:29.009000+0000","last_peered":"2024-04-24T23:14:29.009000+0000","last_clean":"2024-04-24T23:14:29.009000+0000","last_became_active":"2024-04-24T23:12:28.009700+0000","last_became_peered":"2024-04-24T23:12:28.009700+0000","last_unstale":"2024-04-24T23:14:29.009000+0000","last_undegraded":"2024-04-24T23:14:29.009000+0000","last_fullsized":"2024-04-24T23:14:29.009000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:26.172386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.934326+0000","last_change":"2024-04-24T23:14:28.934326+0000","last_active":"2024-04-24T23:14:28.934326+0000","last_peered":"2024-04-24T23:14:28.934326+0000","last_clean":"2024-04-24T23:14:28.934326+0000","last_became_active":"2024-04-24T23:14:28.933888+0000","last_became_peered":"2024-04-24T23:14:28.933888+0000","last_unstale":"2024-04-24T23:14:28.934326+0000","last_undegraded":"2024-04-24T23:14:28.934326+0000","last_fullsized":"2024-04-24T23:14:28.934326+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:04.909941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295666+0000","last_change":"2024-04-24T23:12:28.004140+0000","last_active":"2024-04-24T23:14:29.295666+0000","last_peered":"2024-04-24T23:14:29.295666+0000","last_clean":"2024-04-24T23:14:29.295666+0000","last_became_active":"2024-04-24T23:12:28.003794+0000","last_became_peered":"2024-04-24T23:12:28.003794+0000","last_unstale":"2024-04-24T23:14:29.295666+0000","last_undegraded":"2024-04-24T23:14:29.295666+0000","last_fullsized":"2024-04-24T23:14:29.295666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:02.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295921+0000","last_change":"2024-04-24T23:12:28.006834+0000","last_active":"2024-04-24T23:14:29.295921+0000","last_peered":"2024-04-24T23:14:29.295921+0000","last_clean":"2024-04-24T23:14:29.295921+0000","last_became_active":"2024-04-24T23:12:28.006504+0000","last_became_peered":"2024-04-24T23:12:28.006504+0000","last_unstale":"2024-04-24T23:14:29.295921+0000","last_undegraded":"2024-04-24T23:14:29.295921+0000","last_fullsized":"2024-04-24T23:14:29.295921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:21.194464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.514939+0000","last_change":"2024-04-24T23:12:28.013132+0000","last_active":"2024-04-24T23:14:29.514939+0000","last_peered":"2024-04-24T23:14:29.514939+0000","last_clean":"2024-04-24T23:14:29.514939+0000","last_became_active":"2024-04-24T23:12:28.012751+0000","last_became_peered":"2024-04-24T23:12:28.012751+0000","last_unstale":"2024-04-24T23:14:29.514939+0000","last_undegraded":"2024-04-24T23:14:29.514939+0000","last_fullsized":"2024-04-24T23:14:29.514939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.942985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515426+0000","last_change":"2024-04-24T23:12:28.013719+0000","last_active":"2024-04-24T23:14:29.515426+0000","last_peered":"2024-04-24T23:14:29.515426+0000","last_clean":"2024-04-24T23:14:29.515426+0000","last_became_active":"2024-04-24T23:12:28.013108+0000","last_became_peered":"2024-04-24T23:12:28.013108+0000","last_unstale":"2024-04-24T23:14:29.515426+0000","last_undegraded":"2024-04-24T23:14:29.515426+0000","last_fullsized":"2024-04-24T23:14:29.515426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:38.248867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932637+0000","last_change":"2024-04-24T23:12:28.012386+0000","last_active":"2024-04-24T23:14:28.932637+0000","last_peered":"2024-04-24T23:14:28.932637+0000","last_clean":"2024-04-24T23:14:28.932637+0000","last_became_active":"2024-04-24T23:12:28.011340+0000","last_became_peered":"2024-04-24T23:12:28.011340+0000","last_unstale":"2024-04-24T23:14:28.932637+0000","last_undegraded":"2024-04-24T23:14:28.932637+0000","last_fullsized":"2024-04-24T23:14:28.932637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:48.115600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930262+0000","last_change":"2024-04-24T23:12:28.004033+0000","last_active":"2024-04-24T23:14:28.930262+0000","last_peered":"2024-04-24T23:14:28.930262+0000","last_clean":"2024-04-24T23:14:28.930262+0000","last_became_active":"2024-04-24T23:12:28.003648+0000","last_became_peered":"2024-04-24T23:12:28.003648+0000","last_unstale":"2024-04-24T23:14:28.930262+0000","last_undegraded":"2024-04-24T23:14:28.930262+0000","last_fullsized":"2024-04-24T23:14:28.930262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:13.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.297170+0000","last_change":"2024-04-24T23:14:29.297170+0000","last_active":"2024-04-24T23:14:29.297170+0000","last_peered":"2024-04-24T23:14:29.297170+0000","last_clean":"2024-04-24T23:14:29.297170+0000","last_became_active":"2024-04-24T23:14:29.296794+0000","last_became_peered":"2024-04-24T23:14:29.296794+0000","last_unstale":"2024-04-24T23:14:29.297170+0000","last_undegraded":"2024-04-24T23:14:29.297170+0000","last_fullsized":"2024-04-24T23:14:29.297170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:19:14.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":158,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:46.030498+0000","last_change":"2024-04-24T23:12:27.005325+0000","last_active":"2024-04-24T23:14:46.030498+0000","last_peered":"2024-04-24T23:14:46.030498+0000","last_clean":"2024-04-24T23:14:46.030498+0000","last_became_active":"2024-04-24T23:12:27.004627+0000","last_became_peered":"2024-04-24T23:12:27.004627+0000","last_unstale":"2024-04-24T23:14:46.030498+0000","last_undegraded":"2024-04-24T23:14:46.030498+0000","last_fullsized":"2024-04-24T23:14:46.030498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.985647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928981+0000","last_change":"2024-04-24T23:12:28.007872+0000","last_active":"2024-04-24T23:14:28.928981+0000","last_peered":"2024-04-24T23:14:28.928981+0000","last_clean":"2024-04-24T23:14:28.928981+0000","last_became_active":"2024-04-24T23:12:28.007493+0000","last_became_peered":"2024-04-24T23:12:28.007493+0000","last_unstale":"2024-04-24T23:14:28.928981+0000","last_undegraded":"2024-04-24T23:14:28.928981+0000","last_fullsized":"2024-04-24T23:14:28.928981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:26.686831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008190+0000","last_change":"2024-04-24T23:12:32.031138+0000","last_active":"2024-04-24T23:14:29.008190+0000","last_peered":"2024-04-24T23:14:29.008190+0000","last_clean":"2024-04-24T23:14:29.008190+0000","last_became_active":"2024-04-24T23:12:32.030626+0000","last_became_peered":"2024-04-24T23:12:32.030626+0000","last_unstale":"2024-04-24T23:14:29.008190+0000","last_undegraded":"2024-04-24T23:14:29.008190+0000","last_fullsized":"2024-04-24T23:14:29.008190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929177+0000","last_change":"2024-04-24T23:12:26.999954+0000","last_active":"2024-04-24T23:14:28.929177+0000","last_peered":"2024-04-24T23:14:28.929177+0000","last_clean":"2024-04-24T23:14:28.929177+0000","last_became_active":"2024-04-24T23:12:26.999507+0000","last_became_peered":"2024-04-24T23:12:26.999507+0000","last_unstale":"2024-04-24T23:14:28.929177+0000","last_undegraded":"2024-04-24T23:14:28.929177+0000","last_fullsized":"2024-04-24T23:14:28.929177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:23.984002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296210+0000","last_change":"2024-04-24T23:12:28.019342+0000","last_active":"2024-04-24T23:14:29.296210+0000","last_peered":"2024-04-24T23:14:29.296210+0000","last_clean":"2024-04-24T23:14:29.296210+0000","last_became_active":"2024-04-24T23:12:28.018976+0000","last_became_peered":"2024-04-24T23:12:28.018976+0000","last_unstale":"2024-04-24T23:14:29.296210+0000","last_undegraded":"2024-04-24T23:14:29.296210+0000","last_fullsized":"2024-04-24T23:14:29.296210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:18.737495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008391+0000","last_change":"2024-04-24T23:12:32.031015+0000","last_active":"2024-04-24T23:14:29.008391+0000","last_peered":"2024-04-24T23:14:29.008391+0000","last_clean":"2024-04-24T23:14:29.008391+0000","last_became_active":"2024-04-24T23:12:32.030564+0000","last_became_peered":"2024-04-24T23:12:32.030564+0000","last_unstale":"2024-04-24T23:14:29.008391+0000","last_undegraded":"2024-04-24T23:14:29.008391+0000","last_fullsized":"2024-04-24T23:14:29.008391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:12.247397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931822+0000","last_change":"2024-04-24T23:12:27.001229+0000","last_active":"2024-04-24T23:14:28.931822+0000","last_peered":"2024-04-24T23:14:28.931822+0000","last_clean":"2024-04-24T23:14:28.931822+0000","last_became_active":"2024-04-24T23:12:27.000889+0000","last_became_peered":"2024-04-24T23:12:27.000889+0000","last_unstale":"2024-04-24T23:14:28.931822+0000","last_undegraded":"2024-04-24T23:14:28.931822+0000","last_fullsized":"2024-04-24T23:14:28.931822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:39.189828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515092+0000","last_change":"2024-04-24T23:12:28.012309+0000","last_active":"2024-04-24T23:14:29.515092+0000","last_peered":"2024-04-24T23:14:29.515092+0000","last_clean":"2024-04-24T23:14:29.515092+0000","last_became_active":"2024-04-24T23:12:28.011537+0000","last_became_peered":"2024-04-24T23:12:28.011537+0000","last_unstale":"2024-04-24T23:14:29.515092+0000","last_undegraded":"2024-04-24T23:14:29.515092+0000","last_fullsized":"2024-04-24T23:14:29.515092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:38.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008237+0000","last_change":"2024-04-24T23:12:32.030830+0000","last_active":"2024-04-24T23:14:29.008237+0000","last_peered":"2024-04-24T23:14:29.008237+0000","last_clean":"2024-04-24T23:14:29.008237+0000","last_became_active":"2024-04-24T23:12:32.030318+0000","last_became_peered":"2024-04-24T23:12:32.030318+0000","last_unstale":"2024-04-24T23:14:29.008237+0000","last_undegraded":"2024-04-24T23:14:29.008237+0000","last_fullsized":"2024-04-24T23:14:29.008237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:25.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931622+0000","last_change":"2024-04-24T23:12:26.997926+0000","last_active":"2024-04-24T23:14:28.931622+0000","last_peered":"2024-04-24T23:14:28.931622+0000","last_clean":"2024-04-24T23:14:28.931622+0000","last_became_active":"2024-04-24T23:12:26.997552+0000","last_became_peered":"2024-04-24T23:12:26.997552+0000","last_unstale":"2024-04-24T23:14:28.931622+0000","last_undegraded":"2024-04-24T23:14:28.931622+0000","last_fullsized":"2024-04-24T23:14:28.931622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:50.926624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012647+0000","last_change":"2024-04-24T23:13:28.338152+0000","last_active":"2024-04-24T23:14:29.012647+0000","last_peered":"2024-04-24T23:14:29.012647+0000","last_clean":"2024-04-24T23:14:29.012647+0000","last_became_active":"2024-04-24T23:13:28.337728+0000","last_became_peered":"2024-04-24T23:13:28.337728+0000","last_unstale":"2024-04-24T23:14:29.012647+0000","last_undegraded":"2024-04-24T23:14:29.012647+0000","last_fullsized":"2024-04-24T23:14:29.012647+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.179108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929875+0000","last_change":"2024-04-24T23:12:32.021601+0000","last_active":"2024-04-24T23:14:28.929875+0000","last_peered":"2024-04-24T23:14:28.929875+0000","last_clean":"2024-04-24T23:14:28.929875+0000","last_became_active":"2024-04-24T23:12:32.021141+0000","last_became_peered":"2024-04-24T23:12:32.021141+0000","last_unstale":"2024-04-24T23:14:28.929875+0000","last_undegraded":"2024-04-24T23:14:28.929875+0000","last_fullsized":"2024-04-24T23:14:28.929875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:53.795337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'8","reported_seq":161,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.044736+0000","last_change":"2024-04-24T23:12:27.002639+0000","last_active":"2024-04-24T23:14:49.044736+0000","last_peered":"2024-04-24T23:14:49.044736+0000","last_clean":"2024-04-24T23:14:49.044736+0000","last_became_active":"2024-04-24T23:12:27.002295+0000","last_became_peered":"2024-04-24T23:12:27.002295+0000","last_unstale":"2024-04-24T23:14:49.044736+0000","last_undegraded":"2024-04-24T23:14:49.044736+0000","last_fullsized":"2024-04-24T23:14:49.044736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:40.962878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931586+0000","last_change":"2024-04-24T23:12:28.000123+0000","last_active":"2024-04-24T23:14:28.931586+0000","last_peered":"2024-04-24T23:14:28.931586+0000","last_clean":"2024-04-24T23:14:28.931586+0000","last_became_active":"2024-04-24T23:12:27.999778+0000","last_became_peered":"2024-04-24T23:12:27.999778+0000","last_unstale":"2024-04-24T23:14:28.931586+0000","last_undegraded":"2024-04-24T23:14:28.931586+0000","last_fullsized":"2024-04-24T23:14:28.931586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:03.827336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929917+0000","last_change":"2024-04-24T23:12:32.023886+0000","last_active":"2024-04-24T23:14:28.929917+0000","last_peered":"2024-04-24T23:14:28.929917+0000","last_clean":"2024-04-24T23:14:28.929917+0000","last_became_active":"2024-04-24T23:12:32.022796+0000","last_became_peered":"2024-04-24T23:12:32.022796+0000","last_unstale":"2024-04-24T23:14:28.929917+0000","last_undegraded":"2024-04-24T23:14:28.929917+0000","last_fullsized":"2024-04-24T23:14:28.929917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:17.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931226+0000","last_change":"2024-04-24T23:12:26.999210+0000","last_active":"2024-04-24T23:14:28.931226+0000","last_peered":"2024-04-24T23:14:28.931226+0000","last_clean":"2024-04-24T23:14:28.931226+0000","last_became_active":"2024-04-24T23:12:26.998642+0000","last_became_peered":"2024-04-24T23:12:26.998642+0000","last_unstale":"2024-04-24T23:14:28.931226+0000","last_undegraded":"2024-04-24T23:14:28.931226+0000","last_fullsized":"2024-04-24T23:14:28.931226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:39.171955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012866+0000","last_change":"2024-04-24T23:13:27.990737+0000","last_active":"2024-04-24T23:14:29.012866+0000","last_peered":"2024-04-24T23:14:29.012866+0000","last_clean":"2024-04-24T23:14:29.012866+0000","last_became_active":"2024-04-24T23:13:27.990428+0000","last_became_peered":"2024-04-24T23:13:27.990428+0000","last_unstale":"2024-04-24T23:14:29.012866+0000","last_undegraded":"2024-04-24T23:14:29.012866+0000","last_fullsized":"2024-04-24T23:14:29.012866+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:40.163241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515481+0000","last_change":"2024-04-24T23:12:32.028192+0000","last_active":"2024-04-24T23:14:29.515481+0000","last_peered":"2024-04-24T23:14:29.515481+0000","last_clean":"2024-04-24T23:14:29.515481+0000","last_became_active":"2024-04-24T23:12:32.027715+0000","last_became_peered":"2024-04-24T23:12:32.027715+0000","last_unstale":"2024-04-24T23:14:29.515481+0000","last_undegraded":"2024-04-24T23:14:29.515481+0000","last_fullsized":"2024-04-24T23:14:29.515481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:22.577980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931020+0000","last_change":"2024-04-24T23:12:27.000212+0000","last_active":"2024-04-24T23:14:28.931020+0000","last_peered":"2024-04-24T23:14:28.931020+0000","last_clean":"2024-04-24T23:14:28.931020+0000","last_became_active":"2024-04-24T23:12:26.999844+0000","last_became_peered":"2024-04-24T23:12:26.999844+0000","last_unstale":"2024-04-24T23:14:28.931020+0000","last_undegraded":"2024-04-24T23:14:28.931020+0000","last_fullsized":"2024-04-24T23:14:28.931020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:00.837670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928366+0000","last_change":"2024-04-24T23:12:28.002875+0000","last_active":"2024-04-24T23:14:28.928366+0000","last_peered":"2024-04-24T23:14:28.928366+0000","last_clean":"2024-04-24T23:14:28.928366+0000","last_became_active":"2024-04-24T23:12:28.002572+0000","last_became_peered":"2024-04-24T23:12:28.002572+0000","last_unstale":"2024-04-24T23:14:28.928366+0000","last_undegraded":"2024-04-24T23:14:28.928366+0000","last_fullsized":"2024-04-24T23:14:28.928366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:55.931371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296043+0000","last_change":"2024-04-24T23:12:32.024729+0000","last_active":"2024-04-24T23:14:29.296043+0000","last_peered":"2024-04-24T23:14:29.296043+0000","last_clean":"2024-04-24T23:14:29.296043+0000","last_became_active":"2024-04-24T23:12:32.024316+0000","last_became_peered":"2024-04-24T23:12:32.024316+0000","last_unstale":"2024-04-24T23:14:29.296043+0000","last_undegraded":"2024-04-24T23:14:29.296043+0000","last_fullsized":"2024-04-24T23:14:29.296043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:21.668782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012232+0000","last_change":"2024-04-24T23:12:27.003435+0000","last_active":"2024-04-24T23:14:29.012232+0000","last_peered":"2024-04-24T23:14:29.012232+0000","last_clean":"2024-04-24T23:14:29.012232+0000","last_became_active":"2024-04-24T23:12:27.003101+0000","last_became_peered":"2024-04-24T23:12:27.003101+0000","last_unstale":"2024-04-24T23:14:29.012232+0000","last_undegraded":"2024-04-24T23:14:29.012232+0000","last_fullsized":"2024-04-24T23:14:29.012232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:42.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931602+0000","last_change":"2024-04-24T23:12:28.005056+0000","last_active":"2024-04-24T23:14:28.931602+0000","last_peered":"2024-04-24T23:14:28.931602+0000","last_clean":"2024-04-24T23:14:28.931602+0000","last_became_active":"2024-04-24T23:12:28.004725+0000","last_became_peered":"2024-04-24T23:12:28.004725+0000","last_unstale":"2024-04-24T23:14:28.931602+0000","last_undegraded":"2024-04-24T23:14:28.931602+0000","last_fullsized":"2024-04-24T23:14:28.931602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:49.384950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928478+0000","last_change":"2024-04-24T23:12:32.032947+0000","last_active":"2024-04-24T23:14:28.928478+0000","last_peered":"2024-04-24T23:14:28.928478+0000","last_clean":"2024-04-24T23:14:28.928478+0000","last_became_active":"2024-04-24T23:12:32.032560+0000","last_became_peered":"2024-04-24T23:12:32.032560+0000","last_unstale":"2024-04-24T23:14:28.928478+0000","last_undegraded":"2024-04-24T23:14:28.928478+0000","last_fullsized":"2024-04-24T23:14:28.928478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:29.282938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":265,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.034588+0000","last_change":"2024-04-24T23:12:27.000011+0000","last_active":"2024-04-24T23:14:48.034588+0000","last_peered":"2024-04-24T23:14:48.034588+0000","last_clean":"2024-04-24T23:14:48.034588+0000","last_became_active":"2024-04-24T23:12:26.999572+0000","last_became_peered":"2024-04-24T23:12:26.999572+0000","last_unstale":"2024-04-24T23:14:48.034588+0000","last_undegraded":"2024-04-24T23:14:48.034588+0000","last_fullsized":"2024-04-24T23:14:48.034588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:50.453964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515416+0000","last_change":"2024-04-24T23:12:28.012808+0000","last_active":"2024-04-24T23:14:29.515416+0000","last_peered":"2024-04-24T23:14:29.515416+0000","last_clean":"2024-04-24T23:14:29.515416+0000","last_became_active":"2024-04-24T23:12:28.012406+0000","last_became_peered":"2024-04-24T23:12:28.012406+0000","last_unstale":"2024-04-24T23:14:29.515416+0000","last_undegraded":"2024-04-24T23:14:29.515416+0000","last_fullsized":"2024-04-24T23:14:29.515416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:19.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931289+0000","last_change":"2024-04-24T23:12:32.027156+0000","last_active":"2024-04-24T23:14:28.931289+0000","last_peered":"2024-04-24T23:14:28.931289+0000","last_clean":"2024-04-24T23:14:28.931289+0000","last_became_active":"2024-04-24T23:12:32.026743+0000","last_became_peered":"2024-04-24T23:12:32.026743+0000","last_unstale":"2024-04-24T23:14:28.931289+0000","last_undegraded":"2024-04-24T23:14:28.931289+0000","last_fullsized":"2024-04-24T23:14:28.931289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:43.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009591+0000","last_change":"2024-04-24T23:12:27.001832+0000","last_active":"2024-04-24T23:14:29.009591+0000","last_peered":"2024-04-24T23:14:29.009591+0000","last_clean":"2024-04-24T23:14:29.009591+0000","last_became_active":"2024-04-24T23:12:27.001482+0000","last_became_peered":"2024-04-24T23:12:27.001482+0000","last_unstale":"2024-04-24T23:14:29.009591+0000","last_undegraded":"2024-04-24T23:14:29.009591+0000","last_fullsized":"2024-04-24T23:14:29.009591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:28.949767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296224+0000","last_change":"2024-04-24T23:13:27.960551+0000","last_active":"2024-04-24T23:14:29.296224+0000","last_peered":"2024-04-24T23:14:29.296224+0000","last_clean":"2024-04-24T23:14:29.296224+0000","last_became_active":"2024-04-24T23:13:27.960264+0000","last_became_peered":"2024-04-24T23:13:27.960264+0000","last_unstale":"2024-04-24T23:14:29.296224+0000","last_undegraded":"2024-04-24T23:14:29.296224+0000","last_fullsized":"2024-04-24T23:14:29.296224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:38.758688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008555+0000","last_change":"2024-04-24T23:12:32.023007+0000","last_active":"2024-04-24T23:14:29.008555+0000","last_peered":"2024-04-24T23:14:29.008555+0000","last_clean":"2024-04-24T23:14:29.008555+0000","last_became_active":"2024-04-24T23:12:32.021324+0000","last_became_peered":"2024-04-24T23:12:32.021324+0000","last_unstale":"2024-04-24T23:14:29.008555+0000","last_undegraded":"2024-04-24T23:14:29.008555+0000","last_fullsized":"2024-04-24T23:14:29.008555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:41.542778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012520+0000","last_change":"2024-04-24T23:12:26.990562+0000","last_active":"2024-04-24T23:14:29.012520+0000","last_peered":"2024-04-24T23:14:29.012520+0000","last_clean":"2024-04-24T23:14:29.012520+0000","last_became_active":"2024-04-24T23:12:26.990146+0000","last_became_peered":"2024-04-24T23:12:26.990146+0000","last_unstale":"2024-04-24T23:14:29.012520+0000","last_undegraded":"2024-04-24T23:14:29.012520+0000","last_fullsized":"2024-04-24T23:14:29.012520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.621051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515294+0000","last_change":"2024-04-24T23:12:28.000954+0000","last_active":"2024-04-24T23:14:29.515294+0000","last_peered":"2024-04-24T23:14:29.515294+0000","last_clean":"2024-04-24T23:14:29.515294+0000","last_became_active":"2024-04-24T23:12:28.000610+0000","last_became_peered":"2024-04-24T23:12:28.000610+0000","last_unstale":"2024-04-24T23:14:29.515294+0000","last_undegraded":"2024-04-24T23:14:29.515294+0000","last_fullsized":"2024-04-24T23:14:29.515294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:32.585919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.516500+0000","last_change":"2024-04-24T23:14:29.516500+0000","last_active":"2024-04-24T23:14:29.516500+0000","last_peered":"2024-04-24T23:14:29.516500+0000","last_clean":"2024-04-24T23:14:29.516500+0000","last_became_active":"2024-04-24T23:14:29.516159+0000","last_became_peered":"2024-04-24T23:14:29.516159+0000","last_unstale":"2024-04-24T23:14:29.516500+0000","last_undegraded":"2024-04-24T23:14:29.516500+0000","last_fullsized":"2024-04-24T23:14:29.516500+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:11.290978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":312,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.043073+0000","last_change":"2024-04-24T23:12:26.996719+0000","last_active":"2024-04-24T23:14:48.043073+0000","last_peered":"2024-04-24T23:14:48.043073+0000","last_clean":"2024-04-24T23:14:48.043073+0000","last_became_active":"2024-04-24T23:12:26.996360+0000","last_became_peered":"2024-04-24T23:12:26.996360+0000","last_unstale":"2024-04-24T23:14:48.043073+0000","last_undegraded":"2024-04-24T23:14:48.043073+0000","last_fullsized":"2024-04-24T23:14:48.043073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.526902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63913,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":183,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932164+0000","last_change":"2024-04-24T23:12:28.002889+0000","last_active":"2024-04-24T23:14:28.932164+0000","last_peered":"2024-04-24T23:14:28.932164+0000","last_clean":"2024-04-24T23:14:28.932164+0000","last_became_active":"2024-04-24T23:12:28.002540+0000","last_became_peered":"2024-04-24T23:12:28.002540+0000","last_unstale":"2024-04-24T23:14:28.932164+0000","last_undegraded":"2024-04-24T23:14:28.932164+0000","last_fullsized":"2024-04-24T23:14:28.932164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:57.792628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012875+0000","last_change":"2024-04-24T23:12:32.031160+0000","last_active":"2024-04-24T23:14:29.012875+0000","last_peered":"2024-04-24T23:14:29.012875+0000","last_clean":"2024-04-24T23:14:29.012875+0000","last_became_active":"2024-04-24T23:12:32.030719+0000","last_became_peered":"2024-04-24T23:12:32.030719+0000","last_unstale":"2024-04-24T23:14:29.012875+0000","last_undegraded":"2024-04-24T23:14:29.012875+0000","last_fullsized":"2024-04-24T23:14:29.012875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:45.648011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515047+0000","last_change":"2024-04-24T23:13:27.967294+0000","last_active":"2024-04-24T23:14:29.515047+0000","last_peered":"2024-04-24T23:14:29.515047+0000","last_clean":"2024-04-24T23:14:29.515047+0000","last_became_active":"2024-04-24T23:13:27.958064+0000","last_became_peered":"2024-04-24T23:13:27.958064+0000","last_unstale":"2024-04-24T23:14:29.515047+0000","last_undegraded":"2024-04-24T23:14:29.515047+0000","last_fullsized":"2024-04-24T23:14:29.515047+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:08.824502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931284+0000","last_change":"2024-04-24T23:12:28.003211+0000","last_active":"2024-04-24T23:14:28.931284+0000","last_peered":"2024-04-24T23:14:28.931284+0000","last_clean":"2024-04-24T23:14:28.931284+0000","last_became_active":"2024-04-24T23:12:28.002897+0000","last_became_peered":"2024-04-24T23:12:28.002897+0000","last_unstale":"2024-04-24T23:14:28.931284+0000","last_undegraded":"2024-04-24T23:14:28.931284+0000","last_fullsized":"2024-04-24T23:14:28.931284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.187239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515101+0000","last_change":"2024-04-24T23:12:32.028857+0000","last_active":"2024-04-24T23:14:29.515101+0000","last_peered":"2024-04-24T23:14:29.515101+0000","last_clean":"2024-04-24T23:14:29.515101+0000","last_became_active":"2024-04-24T23:12:32.028455+0000","last_became_peered":"2024-04-24T23:12:32.028455+0000","last_unstale":"2024-04-24T23:14:29.515101+0000","last_undegraded":"2024-04-24T23:14:29.515101+0000","last_fullsized":"2024-04-24T23:14:29.515101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:23.073895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012600+0000","last_change":"2024-04-24T23:12:27.003525+0000","last_active":"2024-04-24T23:14:29.012600+0000","last_peered":"2024-04-24T23:14:29.012600+0000","last_clean":"2024-04-24T23:14:29.012600+0000","last_became_active":"2024-04-24T23:12:27.003203+0000","last_became_peered":"2024-04-24T23:12:27.003203+0000","last_unstale":"2024-04-24T23:14:29.012600+0000","last_undegraded":"2024-04-24T23:14:29.012600+0000","last_fullsized":"2024-04-24T23:14:29.012600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:18.521925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009356+0000","last_change":"2024-04-24T23:12:28.017111+0000","last_active":"2024-04-24T23:14:29.009356+0000","last_peered":"2024-04-24T23:14:29.009356+0000","last_clean":"2024-04-24T23:14:29.009356+0000","last_became_active":"2024-04-24T23:12:28.016725+0000","last_became_peered":"2024-04-24T23:12:28.016725+0000","last_unstale":"2024-04-24T23:14:29.009356+0000","last_undegraded":"2024-04-24T23:14:29.009356+0000","last_fullsized":"2024-04-24T23:14:29.009356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:25.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.297405+0000","last_change":"2024-04-24T23:14:29.297405+0000","last_active":"2024-04-24T23:14:29.297405+0000","last_peered":"2024-04-24T23:14:29.297405+0000","last_clean":"2024-04-24T23:14:29.297405+0000","last_became_active":"2024-04-24T23:14:29.296936+0000","last_became_peered":"2024-04-24T23:14:29.296936+0000","last_unstale":"2024-04-24T23:14:29.297405+0000","last_undegraded":"2024-04-24T23:14:29.297405+0000","last_fullsized":"2024-04-24T23:14:29.297405+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:53.458679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:50.035879+0000","last_change":"2024-04-24T23:12:27.002797+0000","last_active":"2024-04-24T23:14:50.035879+0000","last_peered":"2024-04-24T23:14:50.035879+0000","last_clean":"2024-04-24T23:14:50.035879+0000","last_became_active":"2024-04-24T23:12:27.002086+0000","last_became_peered":"2024-04-24T23:12:27.002086+0000","last_unstale":"2024-04-24T23:14:50.035879+0000","last_undegraded":"2024-04-24T23:14:50.035879+0000","last_fullsized":"2024-04-24T23:14:50.035879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:55.025453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009610+0000","last_change":"2024-04-24T23:13:27.955130+0000","last_active":"2024-04-24T23:14:29.009610+0000","last_peered":"2024-04-24T23:14:29.009610+0000","last_clean":"2024-04-24T23:14:29.009610+0000","last_became_active":"2024-04-24T23:13:27.954784+0000","last_became_peered":"2024-04-24T23:13:27.954784+0000","last_unstale":"2024-04-24T23:14:29.009610+0000","last_undegraded":"2024-04-24T23:14:29.009610+0000","last_fullsized":"2024-04-24T23:14:29.009610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:10.574771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931931+0000","last_change":"2024-04-24T23:12:32.029104+0000","last_active":"2024-04-24T23:14:28.931931+0000","last_peered":"2024-04-24T23:14:28.931931+0000","last_clean":"2024-04-24T23:14:28.931931+0000","last_became_active":"2024-04-24T23:12:32.028704+0000","last_became_peered":"2024-04-24T23:12:32.028704+0000","last_unstale":"2024-04-24T23:14:28.931931+0000","last_undegraded":"2024-04-24T23:14:28.931931+0000","last_fullsized":"2024-04-24T23:14:28.931931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:33.200884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.514991+0000","last_change":"2024-04-24T23:13:27.976835+0000","last_active":"2024-04-24T23:14:29.514991+0000","last_peered":"2024-04-24T23:14:29.514991+0000","last_clean":"2024-04-24T23:14:29.514991+0000","last_became_active":"2024-04-24T23:13:27.959684+0000","last_became_peered":"2024-04-24T23:13:27.959684+0000","last_unstale":"2024-04-24T23:14:29.514991+0000","last_undegraded":"2024-04-24T23:14:29.514991+0000","last_fullsized":"2024-04-24T23:14:29.514991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:24.327964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012405+0000","last_change":"2024-04-24T23:12:28.010510+0000","last_active":"2024-04-24T23:14:29.012405+0000","last_peered":"2024-04-24T23:14:29.012405+0000","last_clean":"2024-04-24T23:14:29.012405+0000","last_became_active":"2024-04-24T23:12:28.009244+0000","last_became_peered":"2024-04-24T23:12:28.009244+0000","last_unstale":"2024-04-24T23:14:29.012405+0000","last_undegraded":"2024-04-24T23:14:29.012405+0000","last_fullsized":"2024-04-24T23:14:29.012405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.146563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012453+0000","last_change":"2024-04-24T23:12:32.031156+0000","last_active":"2024-04-24T23:14:29.012453+0000","last_peered":"2024-04-24T23:14:29.012453+0000","last_clean":"2024-04-24T23:14:29.012453+0000","last_became_active":"2024-04-24T23:12:32.030653+0000","last_became_peered":"2024-04-24T23:12:32.030653+0000","last_unstale":"2024-04-24T23:14:29.012453+0000","last_undegraded":"2024-04-24T23:14:29.012453+0000","last_fullsized":"2024-04-24T23:14:29.012453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:48.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932509+0000","last_change":"2024-04-24T23:12:26.998003+0000","last_active":"2024-04-24T23:14:28.932509+0000","last_peered":"2024-04-24T23:14:28.932509+0000","last_clean":"2024-04-24T23:14:28.932509+0000","last_became_active":"2024-04-24T23:12:26.997661+0000","last_became_peered":"2024-04-24T23:12:26.997661+0000","last_unstale":"2024-04-24T23:14:28.932509+0000","last_undegraded":"2024-04-24T23:14:28.932509+0000","last_fullsized":"2024-04-24T23:14:28.932509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:17.995397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515371+0000","last_change":"2024-04-24T23:12:28.004822+0000","last_active":"2024-04-24T23:14:29.515371+0000","last_peered":"2024-04-24T23:14:29.515371+0000","last_clean":"2024-04-24T23:14:29.515371+0000","last_became_active":"2024-04-24T23:12:28.004334+0000","last_became_peered":"2024-04-24T23:12:28.004334+0000","last_unstale":"2024-04-24T23:14:29.515371+0000","last_undegraded":"2024-04-24T23:14:29.515371+0000","last_fullsized":"2024-04-24T23:14:29.515371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.717796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931518+0000","last_change":"2024-04-24T23:12:32.085309+0000","last_active":"2024-04-24T23:14:28.931518+0000","last_peered":"2024-04-24T23:14:28.931518+0000","last_clean":"2024-04-24T23:14:28.931518+0000","last_became_active":"2024-04-24T23:12:32.084892+0000","last_became_peered":"2024-04-24T23:12:32.084892+0000","last_unstale":"2024-04-24T23:14:28.931518+0000","last_undegraded":"2024-04-24T23:14:28.931518+0000","last_fullsized":"2024-04-24T23:14:28.931518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:20.318901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932253+0000","last_change":"2024-04-24T23:12:27.005096+0000","last_active":"2024-04-24T23:14:28.932253+0000","last_peered":"2024-04-24T23:14:28.932253+0000","last_clean":"2024-04-24T23:14:28.932253+0000","last_became_active":"2024-04-24T23:12:27.004699+0000","last_became_peered":"2024-04-24T23:12:27.004699+0000","last_unstale":"2024-04-24T23:14:28.932253+0000","last_undegraded":"2024-04-24T23:14:28.932253+0000","last_fullsized":"2024-04-24T23:14:28.932253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:58.225361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931600+0000","last_change":"2024-04-24T23:12:28.012230+0000","last_active":"2024-04-24T23:14:28.931600+0000","last_peered":"2024-04-24T23:14:28.931600+0000","last_clean":"2024-04-24T23:14:28.931600+0000","last_became_active":"2024-04-24T23:12:28.011900+0000","last_became_peered":"2024-04-24T23:12:28.011900+0000","last_unstale":"2024-04-24T23:14:28.931600+0000","last_undegraded":"2024-04-24T23:14:28.931600+0000","last_fullsized":"2024-04-24T23:14:28.931600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.780548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932162+0000","last_change":"2024-04-24T23:12:32.026388+0000","last_active":"2024-04-24T23:14:28.932162+0000","last_peered":"2024-04-24T23:14:28.932162+0000","last_clean":"2024-04-24T23:14:28.932162+0000","last_became_active":"2024-04-24T23:12:32.025994+0000","last_became_peered":"2024-04-24T23:12:32.025994+0000","last_unstale":"2024-04-24T23:14:28.932162+0000","last_undegraded":"2024-04-24T23:14:28.932162+0000","last_fullsized":"2024-04-24T23:14:28.932162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:13.062781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":153,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.034564+0000","last_change":"2024-04-24T23:12:27.005080+0000","last_active":"2024-04-24T23:14:49.034564+0000","last_peered":"2024-04-24T23:14:49.034564+0000","last_clean":"2024-04-24T23:14:49.034564+0000","last_became_active":"2024-04-24T23:12:27.004740+0000","last_became_peered":"2024-04-24T23:12:27.004740+0000","last_unstale":"2024-04-24T23:14:49.034564+0000","last_undegraded":"2024-04-24T23:14:49.034564+0000","last_fullsized":"2024-04-24T23:14:49.034564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:22.502399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931619+0000","last_change":"2024-04-24T23:12:28.008016+0000","last_active":"2024-04-24T23:14:28.931619+0000","last_peered":"2024-04-24T23:14:28.931619+0000","last_clean":"2024-04-24T23:14:28.931619+0000","last_became_active":"2024-04-24T23:12:28.007139+0000","last_became_peered":"2024-04-24T23:12:28.007139+0000","last_unstale":"2024-04-24T23:14:28.931619+0000","last_undegraded":"2024-04-24T23:14:28.931619+0000","last_fullsized":"2024-04-24T23:14:28.931619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.698193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932114+0000","last_change":"2024-04-24T23:12:32.085290+0000","last_active":"2024-04-24T23:14:28.932114+0000","last_peered":"2024-04-24T23:14:28.932114+0000","last_clean":"2024-04-24T23:14:28.932114+0000","last_became_active":"2024-04-24T23:12:32.084852+0000","last_became_peered":"2024-04-24T23:12:32.084852+0000","last_unstale":"2024-04-24T23:14:28.932114+0000","last_undegraded":"2024-04-24T23:14:28.932114+0000","last_fullsized":"2024-04-24T23:14:28.932114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.753633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012753+0000","last_change":"2024-04-24T23:12:28.007552+0000","last_active":"2024-04-24T23:14:29.012753+0000","last_peered":"2024-04-24T23:14:29.012753+0000","last_clean":"2024-04-24T23:14:29.012753+0000","last_became_active":"2024-04-24T23:12:28.007196+0000","last_became_peered":"2024-04-24T23:12:28.007196+0000","last_unstale":"2024-04-24T23:14:29.012753+0000","last_undegraded":"2024-04-24T23:14:29.012753+0000","last_fullsized":"2024-04-24T23:14:29.012753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:15.695485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.032904+0000","last_change":"2024-04-24T23:12:26.990814+0000","last_active":"2024-04-24T23:14:48.032904+0000","last_peered":"2024-04-24T23:14:48.032904+0000","last_clean":"2024-04-24T23:14:48.032904+0000","last_became_active":"2024-04-24T23:12:26.990476+0000","last_became_peered":"2024-04-24T23:12:26.990476+0000","last_unstale":"2024-04-24T23:14:48.032904+0000","last_undegraded":"2024-04-24T23:14:48.032904+0000","last_fullsized":"2024-04-24T23:14:48.032904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:32.645770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009649+0000","last_change":"2024-04-24T23:12:32.022039+0000","last_active":"2024-04-24T23:14:29.009649+0000","last_peered":"2024-04-24T23:14:29.009649+0000","last_clean":"2024-04-24T23:14:29.009649+0000","last_became_active":"2024-04-24T23:12:32.021514+0000","last_became_peered":"2024-04-24T23:12:32.021514+0000","last_unstale":"2024-04-24T23:14:29.009649+0000","last_undegraded":"2024-04-24T23:14:29.009649+0000","last_fullsized":"2024-04-24T23:14:29.009649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:42.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928756+0000","last_change":"2024-04-24T23:12:28.008676+0000","last_active":"2024-04-24T23:14:28.928756+0000","last_peered":"2024-04-24T23:14:28.928756+0000","last_clean":"2024-04-24T23:14:28.928756+0000","last_became_active":"2024-04-24T23:12:28.008376+0000","last_became_peered":"2024-04-24T23:12:28.008376+0000","last_unstale":"2024-04-24T23:14:28.928756+0000","last_undegraded":"2024-04-24T23:14:28.928756+0000","last_fullsized":"2024-04-24T23:14:28.928756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:15.358332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008483+0000","last_change":"2024-04-24T23:12:26.994952+0000","last_active":"2024-04-24T23:14:29.008483+0000","last_peered":"2024-04-24T23:14:29.008483+0000","last_clean":"2024-04-24T23:14:29.008483+0000","last_became_active":"2024-04-24T23:12:26.994556+0000","last_became_peered":"2024-04-24T23:12:26.994556+0000","last_unstale":"2024-04-24T23:14:29.008483+0000","last_undegraded":"2024-04-24T23:14:29.008483+0000","last_fullsized":"2024-04-24T23:14:29.008483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.257427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009006+0000","last_change":"2024-04-24T23:11:16.721876+0000","last_active":"2024-04-24T23:14:29.009006+0000","last_peered":"2024-04-24T23:14:29.009006+0000","last_clean":"2024-04-24T23:14:29.009006+0000","last_became_active":"2024-04-24T23:11:16.721291+0000","last_became_peered":"2024-04-24T23:11:16.721291+0000","last_unstale":"2024-04-24T23:14:29.009006+0000","last_undegraded":"2024-04-24T23:14:29.009006+0000","last_fullsized":"2024-04-24T23:14:29.009006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_clean_scrub_stamp":"2024-04-24T23:07:51.257753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.458939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.013052+0000","last_change":"2024-04-24T23:12:32.085905+0000","last_active":"2024-04-24T23:14:29.013052+0000","last_peered":"2024-04-24T23:14:29.013052+0000","last_clean":"2024-04-24T23:14:29.013052+0000","last_became_active":"2024-04-24T23:12:32.085374+0000","last_became_peered":"2024-04-24T23:12:32.085374+0000","last_unstale":"2024-04-24T23:14:29.013052+0000","last_undegraded":"2024-04-24T23:14:29.013052+0000","last_fullsized":"2024-04-24T23:14:29.013052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:16.290651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931162+0000","last_change":"2024-04-24T23:12:27.999880+0000","last_active":"2024-04-24T23:14:28.931162+0000","last_peered":"2024-04-24T23:14:28.931162+0000","last_clean":"2024-04-24T23:14:28.931162+0000","last_became_active":"2024-04-24T23:12:27.999543+0000","last_became_peered":"2024-04-24T23:12:27.999543+0000","last_unstale":"2024-04-24T23:14:28.931162+0000","last_undegraded":"2024-04-24T23:14:28.931162+0000","last_fullsized":"2024-04-24T23:14:28.931162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:44.758968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009273+0000","last_change":"2024-04-24T23:13:28.341009+0000","last_active":"2024-04-24T23:14:29.009273+0000","last_peered":"2024-04-24T23:14:29.009273+0000","last_clean":"2024-04-24T23:14:29.009273+0000","last_became_active":"2024-04-24T23:13:28.338192+0000","last_became_peered":"2024-04-24T23:13:28.338192+0000","last_unstale":"2024-04-24T23:14:29.009273+0000","last_undegraded":"2024-04-24T23:14:29.009273+0000","last_fullsized":"2024-04-24T23:14:29.009273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:11.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515225+0000","last_change":"2024-04-24T23:12:32.024952+0000","last_active":"2024-04-24T23:14:29.515225+0000","last_peered":"2024-04-24T23:14:29.515225+0000","last_clean":"2024-04-24T23:14:29.515225+0000","last_became_active":"2024-04-24T23:12:32.024404+0000","last_became_peered":"2024-04-24T23:12:32.024404+0000","last_unstale":"2024-04-24T23:14:29.515225+0000","last_undegraded":"2024-04-24T23:14:29.515225+0000","last_fullsized":"2024-04-24T23:14:29.515225+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.377794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932193+0000","last_change":"2024-04-24T23:12:27.007191+0000","last_active":"2024-04-24T23:14:28.932193+0000","last_peered":"2024-04-24T23:14:28.932193+0000","last_clean":"2024-04-24T23:14:28.932193+0000","last_became_active":"2024-04-24T23:12:27.006867+0000","last_became_peered":"2024-04-24T23:12:27.006867+0000","last_unstale":"2024-04-24T23:14:28.932193+0000","last_undegraded":"2024-04-24T23:14:28.932193+0000","last_fullsized":"2024-04-24T23:14:28.932193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:58.112293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932401+0000","last_change":"2024-04-24T23:12:27.996322+0000","last_active":"2024-04-24T23:14:28.932401+0000","last_peered":"2024-04-24T23:14:28.932401+0000","last_clean":"2024-04-24T23:14:28.932401+0000","last_became_active":"2024-04-24T23:12:27.995991+0000","last_became_peered":"2024-04-24T23:12:27.995991+0000","last_unstale":"2024-04-24T23:14:28.932401+0000","last_undegraded":"2024-04-24T23:14:28.932401+0000","last_fullsized":"2024-04-24T23:14:28.932401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:08.960347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928414+0000","last_change":"2024-04-24T23:12:32.084959+0000","last_active":"2024-04-24T23:14:28.928414+0000","last_peered":"2024-04-24T23:14:28.928414+0000","last_clean":"2024-04-24T23:14:28.928414+0000","last_became_active":"2024-04-24T23:12:32.084553+0000","last_became_peered":"2024-04-24T23:12:32.084553+0000","last_unstale":"2024-04-24T23:14:28.928414+0000","last_undegraded":"2024-04-24T23:14:28.928414+0000","last_fullsized":"2024-04-24T23:14:28.928414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.836337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932585+0000","last_change":"2024-04-24T23:12:27.007742+0000","last_active":"2024-04-24T23:14:28.932585+0000","last_peered":"2024-04-24T23:14:28.932585+0000","last_clean":"2024-04-24T23:14:28.932585+0000","last_became_active":"2024-04-24T23:12:27.007423+0000","last_became_peered":"2024-04-24T23:12:27.007423+0000","last_unstale":"2024-04-24T23:14:28.932585+0000","last_undegraded":"2024-04-24T23:14:28.932585+0000","last_fullsized":"2024-04-24T23:14:28.932585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:39.478356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928898+0000","last_change":"2024-04-24T23:12:28.010961+0000","last_active":"2024-04-24T23:14:28.928898+0000","last_peered":"2024-04-24T23:14:28.928898+0000","last_clean":"2024-04-24T23:14:28.928898+0000","last_became_active":"2024-04-24T23:12:28.010641+0000","last_became_peered":"2024-04-24T23:12:28.010641+0000","last_unstale":"2024-04-24T23:14:28.928898+0000","last_undegraded":"2024-04-24T23:14:28.928898+0000","last_fullsized":"2024-04-24T23:14:28.928898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:43.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929416+0000","last_change":"2024-04-24T23:12:32.023687+0000","last_active":"2024-04-24T23:14:28.929416+0000","last_peered":"2024-04-24T23:14:28.929416+0000","last_clean":"2024-04-24T23:14:28.929416+0000","last_became_active":"2024-04-24T23:12:32.023052+0000","last_became_peered":"2024-04-24T23:12:32.023052+0000","last_unstale":"2024-04-24T23:14:28.929416+0000","last_undegraded":"2024-04-24T23:14:28.929416+0000","last_fullsized":"2024-04-24T23:14:28.929416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:39.854030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008925+0000","last_change":"2024-04-24T23:12:27.009743+0000","last_active":"2024-04-24T23:14:29.008925+0000","last_peered":"2024-04-24T23:14:29.008925+0000","last_clean":"2024-04-24T23:14:29.008925+0000","last_became_active":"2024-04-24T23:12:27.009343+0000","last_became_peered":"2024-04-24T23:12:27.009343+0000","last_unstale":"2024-04-24T23:14:29.008925+0000","last_undegraded":"2024-04-24T23:14:29.008925+0000","last_fullsized":"2024-04-24T23:14:29.008925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:20.387738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296261+0000","last_change":"2024-04-24T23:13:27.961769+0000","last_active":"2024-04-24T23:14:29.296261+0000","last_peered":"2024-04-24T23:14:29.296261+0000","last_clean":"2024-04-24T23:14:29.296261+0000","last_became_active":"2024-04-24T23:13:27.961460+0000","last_became_peered":"2024-04-24T23:13:27.961460+0000","last_unstale":"2024-04-24T23:14:29.296261+0000","last_undegraded":"2024-04-24T23:14:29.296261+0000","last_fullsized":"2024-04-24T23:14:29.296261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:01.012688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009238+0000","last_change":"2024-04-24T23:12:32.086209+0000","last_active":"2024-04-24T23:14:29.009238+0000","last_peered":"2024-04-24T23:14:29.009238+0000","last_clean":"2024-04-24T23:14:29.009238+0000","last_became_active":"2024-04-24T23:12:32.085752+0000","last_became_peered":"2024-04-24T23:12:32.085752+0000","last_unstale":"2024-04-24T23:14:29.009238+0000","last_undegraded":"2024-04-24T23:14:29.009238+0000","last_fullsized":"2024-04-24T23:14:29.009238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:52.120905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515056+0000","last_change":"2024-04-24T23:12:26.996355+0000","last_active":"2024-04-24T23:14:29.515056+0000","last_peered":"2024-04-24T23:14:29.515056+0000","last_clean":"2024-04-24T23:14:29.515056+0000","last_became_active":"2024-04-24T23:12:26.995486+0000","last_became_peered":"2024-04-24T23:12:26.995486+0000","last_unstale":"2024-04-24T23:14:29.515056+0000","last_undegraded":"2024-04-24T23:14:29.515056+0000","last_fullsized":"2024-04-24T23:14:29.515056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:48.981985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931625+0000","last_change":"2024-04-24T23:12:28.004610+0000","last_active":"2024-04-24T23:14:28.931625+0000","last_peered":"2024-04-24T23:14:28.931625+0000","last_clean":"2024-04-24T23:14:28.931625+0000","last_became_active":"2024-04-24T23:12:28.004051+0000","last_became_peered":"2024-04-24T23:12:28.004051+0000","last_unstale":"2024-04-24T23:14:28.931625+0000","last_undegraded":"2024-04-24T23:14:28.931625+0000","last_fullsized":"2024-04-24T23:14:28.931625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:42.656954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.014680+0000","last_change":"2024-04-24T23:14:29.014103+0000","last_active":"2024-04-24T23:14:29.014680+0000","last_peered":"2024-04-24T23:14:29.014680+0000","last_clean":"2024-04-24T23:14:29.014680+0000","last_became_active":"2024-04-24T23:14:29.013472+0000","last_became_peered":"2024-04-24T23:14:29.013472+0000","last_unstale":"2024-04-24T23:14:29.014680+0000","last_undegraded":"2024-04-24T23:14:29.014680+0000","last_fullsized":"2024-04-24T23:14:29.014680+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:37.823011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928576+0000","last_change":"2024-04-24T23:12:27.006357+0000","last_active":"2024-04-24T23:14:28.928576+0000","last_peered":"2024-04-24T23:14:28.928576+0000","last_clean":"2024-04-24T23:14:28.928576+0000","last_became_active":"2024-04-24T23:12:27.006040+0000","last_became_peered":"2024-04-24T23:12:27.006040+0000","last_unstale":"2024-04-24T23:14:28.928576+0000","last_undegraded":"2024-04-24T23:14:28.928576+0000","last_fullsized":"2024-04-24T23:14:28.928576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:27.520663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515601+0000","last_change":"2024-04-24T23:12:28.008909+0000","last_active":"2024-04-24T23:14:29.515601+0000","last_peered":"2024-04-24T23:14:29.515601+0000","last_clean":"2024-04-24T23:14:29.515601+0000","last_became_active":"2024-04-24T23:12:28.008552+0000","last_became_peered":"2024-04-24T23:12:28.008552+0000","last_unstale":"2024-04-24T23:14:29.515601+0000","last_undegraded":"2024-04-24T23:14:29.515601+0000","last_fullsized":"2024-04-24T23:14:29.515601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:20.628234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008834+0000","last_change":"2024-04-24T23:12:32.029720+0000","last_active":"2024-04-24T23:14:29.008834+0000","last_peered":"2024-04-24T23:14:29.008834+0000","last_clean":"2024-04-24T23:14:29.008834+0000","last_became_active":"2024-04-24T23:12:32.029311+0000","last_became_peered":"2024-04-24T23:12:32.029311+0000","last_unstale":"2024-04-24T23:14:29.008834+0000","last_undegraded":"2024-04-24T23:14:29.008834+0000","last_fullsized":"2024-04-24T23:14:29.008834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:37.832102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:40.286863+0000","last_change":"2024-04-24T23:12:26.995552+0000","last_active":"2024-04-24T23:14:40.286863+0000","last_peered":"2024-04-24T23:14:40.286863+0000","last_clean":"2024-04-24T23:14:40.286863+0000","last_became_active":"2024-04-24T23:12:26.994922+0000","last_became_peered":"2024-04-24T23:12:26.994922+0000","last_unstale":"2024-04-24T23:14:40.286863+0000","last_undegraded":"2024-04-24T23:14:40.286863+0000","last_fullsized":"2024-04-24T23:14:40.286863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:17.981344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928695+0000","last_change":"2024-04-24T23:12:28.008802+0000","last_active":"2024-04-24T23:14:28.928695+0000","last_peered":"2024-04-24T23:14:28.928695+0000","last_clean":"2024-04-24T23:14:28.928695+0000","last_became_active":"2024-04-24T23:12:28.008496+0000","last_became_peered":"2024-04-24T23:12:28.008496+0000","last_unstale":"2024-04-24T23:14:28.928695+0000","last_undegraded":"2024-04-24T23:14:28.928695+0000","last_fullsized":"2024-04-24T23:14:28.928695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:54.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008767+0000","last_change":"2024-04-24T23:12:32.085555+0000","last_active":"2024-04-24T23:14:29.008767+0000","last_peered":"2024-04-24T23:14:29.008767+0000","last_clean":"2024-04-24T23:14:29.008767+0000","last_became_active":"2024-04-24T23:12:32.085085+0000","last_became_peered":"2024-04-24T23:12:32.085085+0000","last_unstale":"2024-04-24T23:14:29.008767+0000","last_undegraded":"2024-04-24T23:14:29.008767+0000","last_fullsized":"2024-04-24T23:14:29.008767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:24.563964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932112+0000","last_change":"2024-04-24T23:13:28.342962+0000","last_active":"2024-04-24T23:14:28.932112+0000","last_peered":"2024-04-24T23:14:28.932112+0000","last_clean":"2024-04-24T23:14:28.932112+0000","last_became_active":"2024-04-24T23:13:28.339742+0000","last_became_peered":"2024-04-24T23:13:28.339742+0000","last_unstale":"2024-04-24T23:14:28.932112+0000","last_undegraded":"2024-04-24T23:14:28.932112+0000","last_fullsized":"2024-04-24T23:14:28.932112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:36.910863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009444+0000","last_change":"2024-04-24T23:12:28.014865+0000","last_active":"2024-04-24T23:14:29.009444+0000","last_peered":"2024-04-24T23:14:29.009444+0000","last_clean":"2024-04-24T23:14:29.009444+0000","last_became_active":"2024-04-24T23:12:28.014538+0000","last_became_peered":"2024-04-24T23:12:28.014538+0000","last_unstale":"2024-04-24T23:14:29.009444+0000","last_undegraded":"2024-04-24T23:14:29.009444+0000","last_fullsized":"2024-04-24T23:14:29.009444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:55.306883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515116+0000","last_change":"2024-04-24T23:12:32.024849+0000","last_active":"2024-04-24T23:14:29.515116+0000","last_peered":"2024-04-24T23:14:29.515116+0000","last_clean":"2024-04-24T23:14:29.515116+0000","last_became_active":"2024-04-24T23:12:32.024412+0000","last_became_peered":"2024-04-24T23:12:32.024412+0000","last_unstale":"2024-04-24T23:14:29.515116+0000","last_undegraded":"2024-04-24T23:14:29.515116+0000","last_fullsized":"2024-04-24T23:14:29.515116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:16.884565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.036058+0000","last_change":"2024-04-24T23:12:27.007284+0000","last_active":"2024-04-24T23:14:49.036058+0000","last_peered":"2024-04-24T23:14:49.036058+0000","last_clean":"2024-04-24T23:14:49.036058+0000","last_became_active":"2024-04-24T23:12:27.006916+0000","last_became_peered":"2024-04-24T23:12:27.006916+0000","last_unstale":"2024-04-24T23:14:49.036058+0000","last_undegraded":"2024-04-24T23:14:49.036058+0000","last_fullsized":"2024-04-24T23:14:49.036058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:40.333387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.010029+0000","last_change":"2024-04-24T23:12:28.011603+0000","last_active":"2024-04-24T23:14:29.010029+0000","last_peered":"2024-04-24T23:14:29.010029+0000","last_clean":"2024-04-24T23:14:29.010029+0000","last_became_active":"2024-04-24T23:12:28.011268+0000","last_became_peered":"2024-04-24T23:12:28.011268+0000","last_unstale":"2024-04-24T23:14:29.010029+0000","last_undegraded":"2024-04-24T23:14:29.010029+0000","last_fullsized":"2024-04-24T23:14:29.010029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.693024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295803+0000","last_change":"2024-04-24T23:12:32.033965+0000","last_active":"2024-04-24T23:14:29.295803+0000","last_peered":"2024-04-24T23:14:29.295803+0000","last_clean":"2024-04-24T23:14:29.295803+0000","last_became_active":"2024-04-24T23:12:32.033472+0000","last_became_peered":"2024-04-24T23:12:32.033472+0000","last_unstale":"2024-04-24T23:14:29.295803+0000","last_undegraded":"2024-04-24T23:14:29.295803+0000","last_fullsized":"2024-04-24T23:14:29.295803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:05.243047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012764+0000","last_change":"2024-04-24T23:12:26.991981+0000","last_active":"2024-04-24T23:14:29.012764+0000","last_peered":"2024-04-24T23:14:29.012764+0000","last_clean":"2024-04-24T23:14:29.012764+0000","last_became_active":"2024-04-24T23:12:26.990707+0000","last_became_peered":"2024-04-24T23:12:26.990707+0000","last_unstale":"2024-04-24T23:14:29.012764+0000","last_undegraded":"2024-04-24T23:14:29.012764+0000","last_fullsized":"2024-04-24T23:14:29.012764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:36.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009154+0000","last_change":"2024-04-24T23:12:28.010428+0000","last_active":"2024-04-24T23:14:29.009154+0000","last_peered":"2024-04-24T23:14:29.009154+0000","last_clean":"2024-04-24T23:14:29.009154+0000","last_became_active":"2024-04-24T23:12:28.010096+0000","last_became_peered":"2024-04-24T23:12:28.010096+0000","last_unstale":"2024-04-24T23:14:29.009154+0000","last_undegraded":"2024-04-24T23:14:29.009154+0000","last_fullsized":"2024-04-24T23:14:29.009154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:31.949617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012035+0000","last_change":"2024-04-24T23:12:32.023828+0000","last_active":"2024-04-24T23:14:29.012035+0000","last_peered":"2024-04-24T23:14:29.012035+0000","last_clean":"2024-04-24T23:14:29.012035+0000","last_became_active":"2024-04-24T23:12:32.023437+0000","last_became_peered":"2024-04-24T23:12:32.023437+0000","last_unstale":"2024-04-24T23:14:29.012035+0000","last_undegraded":"2024-04-24T23:14:29.012035+0000","last_fullsized":"2024-04-24T23:14:29.012035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.186860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931862+0000","last_change":"2024-04-24T23:12:26.993711+0000","last_active":"2024-04-24T23:14:28.931862+0000","last_peered":"2024-04-24T23:14:28.931862+0000","last_clean":"2024-04-24T23:14:28.931862+0000","last_became_active":"2024-04-24T23:12:26.993368+0000","last_became_peered":"2024-04-24T23:12:26.993368+0000","last_unstale":"2024-04-24T23:14:28.931862+0000","last_undegraded":"2024-04-24T23:14:28.931862+0000","last_fullsized":"2024-04-24T23:14:28.931862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:58.843747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009225+0000","last_change":"2024-04-24T23:12:28.003026+0000","last_active":"2024-04-24T23:14:29.009225+0000","last_peered":"2024-04-24T23:14:29.009225+0000","last_clean":"2024-04-24T23:14:29.009225+0000","last_became_active":"2024-04-24T23:12:28.002693+0000","last_became_peered":"2024-04-24T23:12:28.002693+0000","last_unstale":"2024-04-24T23:14:29.009225+0000","last_undegraded":"2024-04-24T23:14:29.009225+0000","last_fullsized":"2024-04-24T23:14:29.009225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:30.926251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515390+0000","last_change":"2024-04-24T23:12:28.013414+0000","last_active":"2024-04-24T23:14:29.515390+0000","last_peered":"2024-04-24T23:14:29.515390+0000","last_clean":"2024-04-24T23:14:29.515390+0000","last_became_active":"2024-04-24T23:12:28.013030+0000","last_became_peered":"2024-04-24T23:12:28.013030+0000","last_unstale":"2024-04-24T23:14:29.515390+0000","last_undegraded":"2024-04-24T23:14:29.515390+0000","last_fullsized":"2024-04-24T23:14:29.515390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:48.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930086+0000","last_change":"2024-04-24T23:12:28.004821+0000","last_active":"2024-04-24T23:14:28.930086+0000","last_peered":"2024-04-24T23:14:28.930086+0000","last_clean":"2024-04-24T23:14:28.930086+0000","last_became_active":"2024-04-24T23:12:28.003162+0000","last_became_peered":"2024-04-24T23:12:28.003162+0000","last_unstale":"2024-04-24T23:14:28.930086+0000","last_undegraded":"2024-04-24T23:14:28.930086+0000","last_fullsized":"2024-04-24T23:14:28.930086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:59.109665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929419+0000","last_change":"2024-04-24T23:12:28.002115+0000","last_active":"2024-04-24T23:14:28.929419+0000","last_peered":"2024-04-24T23:14:28.929419+0000","last_clean":"2024-04-24T23:14:28.929419+0000","last_became_active":"2024-04-24T23:12:28.001727+0000","last_became_peered":"2024-04-24T23:12:28.001727+0000","last_unstale":"2024-04-24T23:14:28.929419+0000","last_undegraded":"2024-04-24T23:14:28.929419+0000","last_fullsized":"2024-04-24T23:14:28.929419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:14.348843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295994+0000","last_change":"2024-04-24T23:12:28.007784+0000","last_active":"2024-04-24T23:14:29.295994+0000","last_peered":"2024-04-24T23:14:29.295994+0000","last_clean":"2024-04-24T23:14:29.295994+0000","last_became_active":"2024-04-24T23:12:28.007481+0000","last_became_peered":"2024-04-24T23:12:28.007481+0000","last_unstale":"2024-04-24T23:14:29.295994+0000","last_undegraded":"2024-04-24T23:14:29.295994+0000","last_fullsized":"2024-04-24T23:14:29.295994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:33.753448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932358+0000","last_change":"2024-04-24T23:12:28.009100+0000","last_active":"2024-04-24T23:14:28.932358+0000","last_peered":"2024-04-24T23:14:28.932358+0000","last_clean":"2024-04-24T23:14:28.932358+0000","last_became_active":"2024-04-24T23:12:28.008782+0000","last_became_peered":"2024-04-24T23:12:28.008782+0000","last_unstale":"2024-04-24T23:14:28.932358+0000","last_undegraded":"2024-04-24T23:14:28.932358+0000","last_fullsized":"2024-04-24T23:14:28.932358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:44.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515607+0000","last_change":"2024-04-24T23:12:28.008129+0000","last_active":"2024-04-24T23:14:29.515607+0000","last_peered":"2024-04-24T23:14:29.515607+0000","last_clean":"2024-04-24T23:14:29.515607+0000","last_became_active":"2024-04-24T23:12:28.007819+0000","last_became_peered":"2024-04-24T23:12:28.007819+0000","last_unstale":"2024-04-24T23:14:29.515607+0000","last_undegraded":"2024-04-24T23:14:29.515607+0000","last_fullsized":"2024-04-24T23:14:29.515607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.234705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930578+0000","last_change":"2024-04-24T23:12:27.993188+0000","last_active":"2024-04-24T23:14:28.930578+0000","last_peered":"2024-04-24T23:14:28.930578+0000","last_clean":"2024-04-24T23:14:28.930578+0000","last_became_active":"2024-04-24T23:12:27.992832+0000","last_became_peered":"2024-04-24T23:12:27.992832+0000","last_unstale":"2024-04-24T23:14:28.930578+0000","last_undegraded":"2024-04-24T23:14:28.930578+0000","last_fullsized":"2024-04-24T23:14:28.930578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:58.623164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928983+0000","last_change":"2024-04-24T23:12:28.004099+0000","last_active":"2024-04-24T23:14:28.928983+0000","last_peered":"2024-04-24T23:14:28.928983+0000","last_clean":"2024-04-24T23:14:28.928983+0000","last_became_active":"2024-04-24T23:12:28.003802+0000","last_became_peered":"2024-04-24T23:12:28.003802+0000","last_unstale":"2024-04-24T23:14:28.928983+0000","last_undegraded":"2024-04-24T23:14:28.928983+0000","last_fullsized":"2024-04-24T23:14:28.928983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:25.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012613+0000","last_change":"2024-04-24T23:12:28.007700+0000","last_active":"2024-04-24T23:14:29.012613+0000","last_peered":"2024-04-24T23:14:29.012613+0000","last_clean":"2024-04-24T23:14:29.012613+0000","last_became_active":"2024-04-24T23:12:28.002923+0000","last_became_peered":"2024-04-24T23:12:28.002923+0000","last_unstale":"2024-04-24T23:14:29.012613+0000","last_undegraded":"2024-04-24T23:14:29.012613+0000","last_fullsized":"2024-04-24T23:14:29.012613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:46.736096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928635+0000","last_change":"2024-04-24T23:12:28.002194+0000","last_active":"2024-04-24T23:14:28.928635+0000","last_peered":"2024-04-24T23:14:28.928635+0000","last_clean":"2024-04-24T23:14:28.928635+0000","last_became_active":"2024-04-24T23:12:28.001703+0000","last_became_peered":"2024-04-24T23:12:28.001703+0000","last_unstale":"2024-04-24T23:14:28.928635+0000","last_undegraded":"2024-04-24T23:14:28.928635+0000","last_fullsized":"2024-04-24T23:14:28.928635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:20.887025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515261+0000","last_change":"2024-04-24T23:12:28.015980+0000","last_active":"2024-04-24T23:14:29.515261+0000","last_peered":"2024-04-24T23:14:29.515261+0000","last_clean":"2024-04-24T23:14:29.515261+0000","last_became_active":"2024-04-24T23:12:28.013510+0000","last_became_peered":"2024-04-24T23:12:28.013510+0000","last_unstale":"2024-04-24T23:14:29.515261+0000","last_undegraded":"2024-04-24T23:14:29.515261+0000","last_fullsized":"2024-04-24T23:14:29.515261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:37.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931126+0000","last_change":"2024-04-24T23:12:28.001883+0000","last_active":"2024-04-24T23:14:28.931126+0000","last_peered":"2024-04-24T23:14:28.931126+0000","last_clean":"2024-04-24T23:14:28.931126+0000","last_became_active":"2024-04-24T23:12:28.001499+0000","last_became_peered":"2024-04-24T23:12:28.001499+0000","last_unstale":"2024-04-24T23:14:28.931126+0000","last_undegraded":"2024-04-24T23:14:28.931126+0000","last_fullsized":"2024-04-24T23:14:28.931126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:46.805548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515457+0000","last_change":"2024-04-24T23:12:28.004660+0000","last_active":"2024-04-24T23:14:29.515457+0000","last_peered":"2024-04-24T23:14:29.515457+0000","last_clean":"2024-04-24T23:14:29.515457+0000","last_became_active":"2024-04-24T23:12:28.004328+0000","last_became_peered":"2024-04-24T23:12:28.004328+0000","last_unstale":"2024-04-24T23:14:29.515457+0000","last_undegraded":"2024-04-24T23:14:29.515457+0000","last_fullsized":"2024-04-24T23:14:29.515457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:04.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71175,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1192,"num_read_kb":671,"num_write":254,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":217839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":525158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32908,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712244,"statfs":{"total":95995035648,"available":95961337856,"internally_reserved":0,"allocated":1773568,"data_stored":1175707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":529004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":525174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1122232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":520412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":589669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":521868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":586347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":520120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":524486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1117814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:14:53.078 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-24T23:14:53.078 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T23:14:53.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.211+0000 7f505094c640 1 -- 172.21.15.112:0/242968092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 msgr2=0x7f504c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.211+0000 7f505094c640 1 --2- 172.21.15.112:0/242968092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 0x7f504c101cf0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5040009900 tx=0x7f504002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 -- 172.21.15.112:0/242968092 shutdown_connections 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 --2- 172.21.15.112:0/242968092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c102230 0x7f504c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 --2- 172.21.15.112:0/242968092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 0x7f504c101cf0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 --2- 172.21.15.112:0/242968092 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f504c100680 0x7f504c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 -- 172.21.15.112:0/242968092 >> 172.21.15.112:0/242968092 conn(0x7f504c0fbbf0 msgr2=0x7f504c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 -- 172.21.15.112:0/242968092 shutdown_connections 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 -- 172.21.15.112:0/242968092 wait complete. 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 Processor -- start 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.212+0000 7f505094c640 1 -- start start 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f505094c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c100680 0x7f504c0693a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f505094c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 0x7f504c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f505094c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f504c102230 0x7f504c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f505094c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10c430 con 0x7f504c101870 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f505094c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10c2b0 con 0x7f504c102230 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f505094c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f504c10c5b0 con 0x7f504c100680 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 0x7f504c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 0x7f504c19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39704/0 (socket says 172.21.15.112:39704) 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 -- 172.21.15.112:0/4043108816 learned_addr learned my addr 172.21.15.112:0/4043108816 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f504a59c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c100680 0x7f504c0693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f504ad9d640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f504c102230 0x7f504c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 -- 172.21.15.112:0/4043108816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c100680 msgr2=0x7f504c0693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c100680 0x7f504c0693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 -- 172.21.15.112:0/4043108816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f504c102230 msgr2=0x7f504c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f504c102230 0x7f504c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 -- 172.21.15.112:0/4043108816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c19f7a0 con 0x7f504c101870 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f504ad9d640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f504c102230 0x7f504c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:53.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f5049d9b640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 0x7f504c19c970 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5040009a20 tx=0x7f5040004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f503b7fe640 1 -- 172.21.15.112:0/4043108816 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504003f030 con 0x7f504c101870 2024-04-24T23:14:53.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c19fa30 con 0x7f504c101870 2024-04-24T23:14:53.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f503b7fe640 1 -- 172.21.15.112:0/4043108816 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f5040003d00 con 0x7f504c101870 2024-04-24T23:14:53.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f503b7fe640 1 -- 172.21.15.112:0/4043108816 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040035070 con 0x7f504c101870 2024-04-24T23:14:53.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.213+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c19ff70 con 0x7f504c101870 2024-04-24T23:14:53.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.214+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c107940 con 0x7f504c101870 2024-04-24T23:14:53.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.216+0000 7f503b7fe640 1 -- 172.21.15.112:0/4043108816 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5040042030 con 0x7f504c101870 2024-04-24T23:14:53.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.216+0000 7f503b7fe640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f50340a2c10 0x7f50340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.216+0000 7f504a59c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f50340a2c10 0x7f50340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.216+0000 7f503b7fe640 1 -- 172.21.15.112:0/4043108816 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f50400efe80 con 0x7f504c101870 2024-04-24T23:14:53.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.217+0000 7f504a59c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f50340a2c10 0x7f50340a50d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f504c1016d0 tx=0x7f503c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.220+0000 7f503b7fe640 1 -- 172.21.15.112:0/4043108816 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f50400bc9f0 con 0x7f504c101870 2024-04-24T23:14:53.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:53 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2089425401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:53.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:53 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2617604532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:53.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.391+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 --> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f504c061b80 con 0x7f50340a2c10 2024-04-24T23:14:53.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.399+0000 7f503b7fe640 1 -- 172.21.15.112:0/4043108816 <== mgr.14150 v2:172.21.15.112:6800/2191305617 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340539 (secure 0 0 0) 0x7f504c061b80 con 0x7f50340a2c10 2024-04-24T23:14:53.404 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:53.406 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f50340a2c10 msgr2=0x7f50340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f50340a2c10 0x7f50340a50d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f504c1016d0 tx=0x7f503c00a2d0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 msgr2=0x7f504c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 0x7f504c19c970 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5040009a20 tx=0x7f5040004430 comp rx=0 tx=0).stop 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 shutdown_connections 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f50340a2c10 0x7f50340a50d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f504c102230 0x7f504c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f504c101870 0x7f504c19c970 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 --2- 172.21.15.112:0/4043108816 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f504c100680 0x7f504c0693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 >> 172.21.15.112:0/4043108816 conn(0x7f504c0fbbf0 msgr2=0x7f504c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 shutdown_connections 2024-04-24T23:14:53.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.403+0000 7f505094c640 1 -- 172.21.15.112:0/4043108816 wait complete. 2024-04-24T23:14:53.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2089425401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:53.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:53 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2617604532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:53.415 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-04-24T23:14:52.822084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661777,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1318,"num_read_kb":780,"num_write":437,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1180916,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384764,"kb_used_data":14772,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557060,"statfs":{"total":1151940427776,"available":1151546429440,"internally_reserved":0,"allocated":15126528,"data_stored":8257991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":102,"num_read_kb":52,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003125"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515637+0000","last_change":"2024-04-24T23:12:28.013506+0000","last_active":"2024-04-24T23:14:29.515637+0000","last_peered":"2024-04-24T23:14:29.515637+0000","last_clean":"2024-04-24T23:14:29.515637+0000","last_became_active":"2024-04-24T23:12:28.012652+0000","last_became_peered":"2024-04-24T23:12:28.012652+0000","last_unstale":"2024-04-24T23:14:29.515637+0000","last_undegraded":"2024-04-24T23:14:29.515637+0000","last_fullsized":"2024-04-24T23:14:29.515637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:07:57.420465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296164+0000","last_change":"2024-04-24T23:12:28.004998+0000","last_active":"2024-04-24T23:14:29.296164+0000","last_peered":"2024-04-24T23:14:29.296164+0000","last_clean":"2024-04-24T23:14:29.296164+0000","last_became_active":"2024-04-24T23:12:28.004706+0000","last_became_peered":"2024-04-24T23:12:28.004706+0000","last_unstale":"2024-04-24T23:14:29.296164+0000","last_undegraded":"2024-04-24T23:14:29.296164+0000","last_fullsized":"2024-04-24T23:14:29.296164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.759515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296026+0000","last_change":"2024-04-24T23:12:28.019170+0000","last_active":"2024-04-24T23:14:29.296026+0000","last_peered":"2024-04-24T23:14:29.296026+0000","last_clean":"2024-04-24T23:14:29.296026+0000","last_became_active":"2024-04-24T23:12:28.018815+0000","last_became_peered":"2024-04-24T23:12:28.018815+0000","last_unstale":"2024-04-24T23:14:29.296026+0000","last_undegraded":"2024-04-24T23:14:29.296026+0000","last_fullsized":"2024-04-24T23:14:29.296026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:43.830598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009162+0000","last_change":"2024-04-24T23:12:28.008662+0000","last_active":"2024-04-24T23:14:29.009162+0000","last_peered":"2024-04-24T23:14:29.009162+0000","last_clean":"2024-04-24T23:14:29.009162+0000","last_became_active":"2024-04-24T23:12:28.008282+0000","last_became_peered":"2024-04-24T23:12:28.008282+0000","last_unstale":"2024-04-24T23:14:29.009162+0000","last_undegraded":"2024-04-24T23:14:29.009162+0000","last_fullsized":"2024-04-24T23:14:29.009162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:51:17.558899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931843+0000","last_change":"2024-04-24T23:12:28.005212+0000","last_active":"2024-04-24T23:14:28.931843+0000","last_peered":"2024-04-24T23:14:28.931843+0000","last_clean":"2024-04-24T23:14:28.931843+0000","last_became_active":"2024-04-24T23:12:28.004853+0000","last_became_peered":"2024-04-24T23:12:28.004853+0000","last_unstale":"2024-04-24T23:14:28.931843+0000","last_undegraded":"2024-04-24T23:14:28.931843+0000","last_fullsized":"2024-04-24T23:14:28.931843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:33.650922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931438+0000","last_change":"2024-04-24T23:12:28.004546+0000","last_active":"2024-04-24T23:14:28.931438+0000","last_peered":"2024-04-24T23:14:28.931438+0000","last_clean":"2024-04-24T23:14:28.931438+0000","last_became_active":"2024-04-24T23:12:28.004194+0000","last_became_peered":"2024-04-24T23:12:28.004194+0000","last_unstale":"2024-04-24T23:14:28.931438+0000","last_undegraded":"2024-04-24T23:14:28.931438+0000","last_fullsized":"2024-04-24T23:14:28.931438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:42:57.772271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008625+0000","last_change":"2024-04-24T23:12:28.009436+0000","last_active":"2024-04-24T23:14:29.008625+0000","last_peered":"2024-04-24T23:14:29.008625+0000","last_clean":"2024-04-24T23:14:29.008625+0000","last_became_active":"2024-04-24T23:12:28.008219+0000","last_became_peered":"2024-04-24T23:12:28.008219+0000","last_unstale":"2024-04-24T23:14:29.008625+0000","last_undegraded":"2024-04-24T23:14:29.008625+0000","last_fullsized":"2024-04-24T23:14:29.008625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:26.949475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930928+0000","last_change":"2024-04-24T23:13:27.957345+0000","last_active":"2024-04-24T23:14:28.930928+0000","last_peered":"2024-04-24T23:14:28.930928+0000","last_clean":"2024-04-24T23:14:28.930928+0000","last_became_active":"2024-04-24T23:13:27.956989+0000","last_became_peered":"2024-04-24T23:13:27.956989+0000","last_unstale":"2024-04-24T23:14:28.930928+0000","last_undegraded":"2024-04-24T23:14:28.930928+0000","last_fullsized":"2024-04-24T23:14:28.930928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:42.120100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515468+0000","last_change":"2024-04-24T23:12:28.004948+0000","last_active":"2024-04-24T23:14:29.515468+0000","last_peered":"2024-04-24T23:14:29.515468+0000","last_clean":"2024-04-24T23:14:29.515468+0000","last_became_active":"2024-04-24T23:12:28.004380+0000","last_became_peered":"2024-04-24T23:12:28.004380+0000","last_unstale":"2024-04-24T23:14:29.515468+0000","last_undegraded":"2024-04-24T23:14:29.515468+0000","last_fullsized":"2024-04-24T23:14:29.515468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:53.999210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931753+0000","last_change":"2024-04-24T23:12:28.005192+0000","last_active":"2024-04-24T23:14:28.931753+0000","last_peered":"2024-04-24T23:14:28.931753+0000","last_clean":"2024-04-24T23:14:28.931753+0000","last_became_active":"2024-04-24T23:12:28.004860+0000","last_became_peered":"2024-04-24T23:12:28.004860+0000","last_unstale":"2024-04-24T23:14:28.931753+0000","last_undegraded":"2024-04-24T23:14:28.931753+0000","last_fullsized":"2024-04-24T23:14:28.931753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:21.874408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009736+0000","last_change":"2024-04-24T23:12:28.016098+0000","last_active":"2024-04-24T23:14:29.009736+0000","last_peered":"2024-04-24T23:14:29.009736+0000","last_clean":"2024-04-24T23:14:29.009736+0000","last_became_active":"2024-04-24T23:12:28.015736+0000","last_became_peered":"2024-04-24T23:12:28.015736+0000","last_unstale":"2024-04-24T23:14:29.009736+0000","last_undegraded":"2024-04-24T23:14:29.009736+0000","last_fullsized":"2024-04-24T23:14:29.009736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:40:16.071797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009000+0000","last_change":"2024-04-24T23:12:28.010651+0000","last_active":"2024-04-24T23:14:29.009000+0000","last_peered":"2024-04-24T23:14:29.009000+0000","last_clean":"2024-04-24T23:14:29.009000+0000","last_became_active":"2024-04-24T23:12:28.009700+0000","last_became_peered":"2024-04-24T23:12:28.009700+0000","last_unstale":"2024-04-24T23:14:29.009000+0000","last_undegraded":"2024-04-24T23:14:29.009000+0000","last_fullsized":"2024-04-24T23:14:29.009000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:41:26.172386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.934326+0000","last_change":"2024-04-24T23:14:28.934326+0000","last_active":"2024-04-24T23:14:28.934326+0000","last_peered":"2024-04-24T23:14:28.934326+0000","last_clean":"2024-04-24T23:14:28.934326+0000","last_became_active":"2024-04-24T23:14:28.933888+0000","last_became_peered":"2024-04-24T23:14:28.933888+0000","last_unstale":"2024-04-24T23:14:28.934326+0000","last_undegraded":"2024-04-24T23:14:28.934326+0000","last_fullsized":"2024-04-24T23:14:28.934326+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:04.909941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295666+0000","last_change":"2024-04-24T23:12:28.004140+0000","last_active":"2024-04-24T23:14:29.295666+0000","last_peered":"2024-04-24T23:14:29.295666+0000","last_clean":"2024-04-24T23:14:29.295666+0000","last_became_active":"2024-04-24T23:12:28.003794+0000","last_became_peered":"2024-04-24T23:12:28.003794+0000","last_unstale":"2024-04-24T23:14:29.295666+0000","last_undegraded":"2024-04-24T23:14:29.295666+0000","last_fullsized":"2024-04-24T23:14:29.295666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:02.605583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295921+0000","last_change":"2024-04-24T23:12:28.006834+0000","last_active":"2024-04-24T23:14:29.295921+0000","last_peered":"2024-04-24T23:14:29.295921+0000","last_clean":"2024-04-24T23:14:29.295921+0000","last_became_active":"2024-04-24T23:12:28.006504+0000","last_became_peered":"2024-04-24T23:12:28.006504+0000","last_unstale":"2024-04-24T23:14:29.295921+0000","last_undegraded":"2024-04-24T23:14:29.295921+0000","last_fullsized":"2024-04-24T23:14:29.295921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:21.194464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.514939+0000","last_change":"2024-04-24T23:12:28.013132+0000","last_active":"2024-04-24T23:14:29.514939+0000","last_peered":"2024-04-24T23:14:29.514939+0000","last_clean":"2024-04-24T23:14:29.514939+0000","last_became_active":"2024-04-24T23:12:28.012751+0000","last_became_peered":"2024-04-24T23:12:28.012751+0000","last_unstale":"2024-04-24T23:14:29.514939+0000","last_undegraded":"2024-04-24T23:14:29.514939+0000","last_fullsized":"2024-04-24T23:14:29.514939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:04:49.942985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515426+0000","last_change":"2024-04-24T23:12:28.013719+0000","last_active":"2024-04-24T23:14:29.515426+0000","last_peered":"2024-04-24T23:14:29.515426+0000","last_clean":"2024-04-24T23:14:29.515426+0000","last_became_active":"2024-04-24T23:12:28.013108+0000","last_became_peered":"2024-04-24T23:12:28.013108+0000","last_unstale":"2024-04-24T23:14:29.515426+0000","last_undegraded":"2024-04-24T23:14:29.515426+0000","last_fullsized":"2024-04-24T23:14:29.515426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:38.248867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932637+0000","last_change":"2024-04-24T23:12:28.012386+0000","last_active":"2024-04-24T23:14:28.932637+0000","last_peered":"2024-04-24T23:14:28.932637+0000","last_clean":"2024-04-24T23:14:28.932637+0000","last_became_active":"2024-04-24T23:12:28.011340+0000","last_became_peered":"2024-04-24T23:12:28.011340+0000","last_unstale":"2024-04-24T23:14:28.932637+0000","last_undegraded":"2024-04-24T23:14:28.932637+0000","last_fullsized":"2024-04-24T23:14:28.932637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:09:48.115600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930262+0000","last_change":"2024-04-24T23:12:28.004033+0000","last_active":"2024-04-24T23:14:28.930262+0000","last_peered":"2024-04-24T23:14:28.930262+0000","last_clean":"2024-04-24T23:14:28.930262+0000","last_became_active":"2024-04-24T23:12:28.003648+0000","last_became_peered":"2024-04-24T23:12:28.003648+0000","last_unstale":"2024-04-24T23:14:28.930262+0000","last_undegraded":"2024-04-24T23:14:28.930262+0000","last_fullsized":"2024-04-24T23:14:28.930262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:07:13.686453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.297170+0000","last_change":"2024-04-24T23:14:29.297170+0000","last_active":"2024-04-24T23:14:29.297170+0000","last_peered":"2024-04-24T23:14:29.297170+0000","last_clean":"2024-04-24T23:14:29.297170+0000","last_became_active":"2024-04-24T23:14:29.296794+0000","last_became_peered":"2024-04-24T23:14:29.296794+0000","last_unstale":"2024-04-24T23:14:29.297170+0000","last_undegraded":"2024-04-24T23:14:29.297170+0000","last_fullsized":"2024-04-24T23:14:29.297170+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:19:14.884591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":158,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:46.030498+0000","last_change":"2024-04-24T23:12:27.005325+0000","last_active":"2024-04-24T23:14:46.030498+0000","last_peered":"2024-04-24T23:14:46.030498+0000","last_clean":"2024-04-24T23:14:46.030498+0000","last_became_active":"2024-04-24T23:12:27.004627+0000","last_became_peered":"2024-04-24T23:12:27.004627+0000","last_unstale":"2024-04-24T23:14:46.030498+0000","last_undegraded":"2024-04-24T23:14:46.030498+0000","last_fullsized":"2024-04-24T23:14:46.030498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.985647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928981+0000","last_change":"2024-04-24T23:12:28.007872+0000","last_active":"2024-04-24T23:14:28.928981+0000","last_peered":"2024-04-24T23:14:28.928981+0000","last_clean":"2024-04-24T23:14:28.928981+0000","last_became_active":"2024-04-24T23:12:28.007493+0000","last_became_peered":"2024-04-24T23:12:28.007493+0000","last_unstale":"2024-04-24T23:14:28.928981+0000","last_undegraded":"2024-04-24T23:14:28.928981+0000","last_fullsized":"2024-04-24T23:14:28.928981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:31:26.686831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008190+0000","last_change":"2024-04-24T23:12:32.031138+0000","last_active":"2024-04-24T23:14:29.008190+0000","last_peered":"2024-04-24T23:14:29.008190+0000","last_clean":"2024-04-24T23:14:29.008190+0000","last_became_active":"2024-04-24T23:12:32.030626+0000","last_became_peered":"2024-04-24T23:12:32.030626+0000","last_unstale":"2024-04-24T23:14:29.008190+0000","last_undegraded":"2024-04-24T23:14:29.008190+0000","last_fullsized":"2024-04-24T23:14:29.008190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:37.170187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929177+0000","last_change":"2024-04-24T23:12:26.999954+0000","last_active":"2024-04-24T23:14:28.929177+0000","last_peered":"2024-04-24T23:14:28.929177+0000","last_clean":"2024-04-24T23:14:28.929177+0000","last_became_active":"2024-04-24T23:12:26.999507+0000","last_became_peered":"2024-04-24T23:12:26.999507+0000","last_unstale":"2024-04-24T23:14:28.929177+0000","last_undegraded":"2024-04-24T23:14:28.929177+0000","last_fullsized":"2024-04-24T23:14:28.929177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:23.984002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296210+0000","last_change":"2024-04-24T23:12:28.019342+0000","last_active":"2024-04-24T23:14:29.296210+0000","last_peered":"2024-04-24T23:14:29.296210+0000","last_clean":"2024-04-24T23:14:29.296210+0000","last_became_active":"2024-04-24T23:12:28.018976+0000","last_became_peered":"2024-04-24T23:12:28.018976+0000","last_unstale":"2024-04-24T23:14:29.296210+0000","last_undegraded":"2024-04-24T23:14:29.296210+0000","last_fullsized":"2024-04-24T23:14:29.296210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:18.737495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008391+0000","last_change":"2024-04-24T23:12:32.031015+0000","last_active":"2024-04-24T23:14:29.008391+0000","last_peered":"2024-04-24T23:14:29.008391+0000","last_clean":"2024-04-24T23:14:29.008391+0000","last_became_active":"2024-04-24T23:12:32.030564+0000","last_became_peered":"2024-04-24T23:12:32.030564+0000","last_unstale":"2024-04-24T23:14:29.008391+0000","last_undegraded":"2024-04-24T23:14:29.008391+0000","last_fullsized":"2024-04-24T23:14:29.008391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:12.247397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931822+0000","last_change":"2024-04-24T23:12:27.001229+0000","last_active":"2024-04-24T23:14:28.931822+0000","last_peered":"2024-04-24T23:14:28.931822+0000","last_clean":"2024-04-24T23:14:28.931822+0000","last_became_active":"2024-04-24T23:12:27.000889+0000","last_became_peered":"2024-04-24T23:12:27.000889+0000","last_unstale":"2024-04-24T23:14:28.931822+0000","last_undegraded":"2024-04-24T23:14:28.931822+0000","last_fullsized":"2024-04-24T23:14:28.931822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:06:39.189828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515092+0000","last_change":"2024-04-24T23:12:28.012309+0000","last_active":"2024-04-24T23:14:29.515092+0000","last_peered":"2024-04-24T23:14:29.515092+0000","last_clean":"2024-04-24T23:14:29.515092+0000","last_became_active":"2024-04-24T23:12:28.011537+0000","last_became_peered":"2024-04-24T23:12:28.011537+0000","last_unstale":"2024-04-24T23:14:29.515092+0000","last_undegraded":"2024-04-24T23:14:29.515092+0000","last_fullsized":"2024-04-24T23:14:29.515092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:38.634529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008237+0000","last_change":"2024-04-24T23:12:32.030830+0000","last_active":"2024-04-24T23:14:29.008237+0000","last_peered":"2024-04-24T23:14:29.008237+0000","last_clean":"2024-04-24T23:14:29.008237+0000","last_became_active":"2024-04-24T23:12:32.030318+0000","last_became_peered":"2024-04-24T23:12:32.030318+0000","last_unstale":"2024-04-24T23:14:29.008237+0000","last_undegraded":"2024-04-24T23:14:29.008237+0000","last_fullsized":"2024-04-24T23:14:29.008237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:25.325345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931622+0000","last_change":"2024-04-24T23:12:26.997926+0000","last_active":"2024-04-24T23:14:28.931622+0000","last_peered":"2024-04-24T23:14:28.931622+0000","last_clean":"2024-04-24T23:14:28.931622+0000","last_became_active":"2024-04-24T23:12:26.997552+0000","last_became_peered":"2024-04-24T23:12:26.997552+0000","last_unstale":"2024-04-24T23:14:28.931622+0000","last_undegraded":"2024-04-24T23:14:28.931622+0000","last_fullsized":"2024-04-24T23:14:28.931622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:12:50.926624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012647+0000","last_change":"2024-04-24T23:13:28.338152+0000","last_active":"2024-04-24T23:14:29.012647+0000","last_peered":"2024-04-24T23:14:29.012647+0000","last_clean":"2024-04-24T23:14:29.012647+0000","last_became_active":"2024-04-24T23:13:28.337728+0000","last_became_peered":"2024-04-24T23:13:28.337728+0000","last_unstale":"2024-04-24T23:14:29.012647+0000","last_undegraded":"2024-04-24T23:14:29.012647+0000","last_fullsized":"2024-04-24T23:14:29.012647+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:55:41.179108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929875+0000","last_change":"2024-04-24T23:12:32.021601+0000","last_active":"2024-04-24T23:14:28.929875+0000","last_peered":"2024-04-24T23:14:28.929875+0000","last_clean":"2024-04-24T23:14:28.929875+0000","last_became_active":"2024-04-24T23:12:32.021141+0000","last_became_peered":"2024-04-24T23:12:32.021141+0000","last_unstale":"2024-04-24T23:14:28.929875+0000","last_undegraded":"2024-04-24T23:14:28.929875+0000","last_fullsized":"2024-04-24T23:14:28.929875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:53.795337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'8","reported_seq":161,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.044736+0000","last_change":"2024-04-24T23:12:27.002639+0000","last_active":"2024-04-24T23:14:49.044736+0000","last_peered":"2024-04-24T23:14:49.044736+0000","last_clean":"2024-04-24T23:14:49.044736+0000","last_became_active":"2024-04-24T23:12:27.002295+0000","last_became_peered":"2024-04-24T23:12:27.002295+0000","last_unstale":"2024-04-24T23:14:49.044736+0000","last_undegraded":"2024-04-24T23:14:49.044736+0000","last_fullsized":"2024-04-24T23:14:49.044736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:40.962878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931586+0000","last_change":"2024-04-24T23:12:28.000123+0000","last_active":"2024-04-24T23:14:28.931586+0000","last_peered":"2024-04-24T23:14:28.931586+0000","last_clean":"2024-04-24T23:14:28.931586+0000","last_became_active":"2024-04-24T23:12:27.999778+0000","last_became_peered":"2024-04-24T23:12:27.999778+0000","last_unstale":"2024-04-24T23:14:28.931586+0000","last_undegraded":"2024-04-24T23:14:28.931586+0000","last_fullsized":"2024-04-24T23:14:28.931586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:53:03.827336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929917+0000","last_change":"2024-04-24T23:12:32.023886+0000","last_active":"2024-04-24T23:14:28.929917+0000","last_peered":"2024-04-24T23:14:28.929917+0000","last_clean":"2024-04-24T23:14:28.929917+0000","last_became_active":"2024-04-24T23:12:32.022796+0000","last_became_peered":"2024-04-24T23:12:32.022796+0000","last_unstale":"2024-04-24T23:14:28.929917+0000","last_undegraded":"2024-04-24T23:14:28.929917+0000","last_fullsized":"2024-04-24T23:14:28.929917+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:27:17.515216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931226+0000","last_change":"2024-04-24T23:12:26.999210+0000","last_active":"2024-04-24T23:14:28.931226+0000","last_peered":"2024-04-24T23:14:28.931226+0000","last_clean":"2024-04-24T23:14:28.931226+0000","last_became_active":"2024-04-24T23:12:26.998642+0000","last_became_peered":"2024-04-24T23:12:26.998642+0000","last_unstale":"2024-04-24T23:14:28.931226+0000","last_undegraded":"2024-04-24T23:14:28.931226+0000","last_fullsized":"2024-04-24T23:14:28.931226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:06:39.171955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012866+0000","last_change":"2024-04-24T23:13:27.990737+0000","last_active":"2024-04-24T23:14:29.012866+0000","last_peered":"2024-04-24T23:14:29.012866+0000","last_clean":"2024-04-24T23:14:29.012866+0000","last_became_active":"2024-04-24T23:13:27.990428+0000","last_became_peered":"2024-04-24T23:13:27.990428+0000","last_unstale":"2024-04-24T23:14:29.012866+0000","last_undegraded":"2024-04-24T23:14:29.012866+0000","last_fullsized":"2024-04-24T23:14:29.012866+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:40.163241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515481+0000","last_change":"2024-04-24T23:12:32.028192+0000","last_active":"2024-04-24T23:14:29.515481+0000","last_peered":"2024-04-24T23:14:29.515481+0000","last_clean":"2024-04-24T23:14:29.515481+0000","last_became_active":"2024-04-24T23:12:32.027715+0000","last_became_peered":"2024-04-24T23:12:32.027715+0000","last_unstale":"2024-04-24T23:14:29.515481+0000","last_undegraded":"2024-04-24T23:14:29.515481+0000","last_fullsized":"2024-04-24T23:14:29.515481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:13:22.577980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931020+0000","last_change":"2024-04-24T23:12:27.000212+0000","last_active":"2024-04-24T23:14:28.931020+0000","last_peered":"2024-04-24T23:14:28.931020+0000","last_clean":"2024-04-24T23:14:28.931020+0000","last_became_active":"2024-04-24T23:12:26.999844+0000","last_became_peered":"2024-04-24T23:12:26.999844+0000","last_unstale":"2024-04-24T23:14:28.931020+0000","last_undegraded":"2024-04-24T23:14:28.931020+0000","last_fullsized":"2024-04-24T23:14:28.931020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:00.837670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928366+0000","last_change":"2024-04-24T23:12:28.002875+0000","last_active":"2024-04-24T23:14:28.928366+0000","last_peered":"2024-04-24T23:14:28.928366+0000","last_clean":"2024-04-24T23:14:28.928366+0000","last_became_active":"2024-04-24T23:12:28.002572+0000","last_became_peered":"2024-04-24T23:12:28.002572+0000","last_unstale":"2024-04-24T23:14:28.928366+0000","last_undegraded":"2024-04-24T23:14:28.928366+0000","last_fullsized":"2024-04-24T23:14:28.928366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:25:55.931371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296043+0000","last_change":"2024-04-24T23:12:32.024729+0000","last_active":"2024-04-24T23:14:29.296043+0000","last_peered":"2024-04-24T23:14:29.296043+0000","last_clean":"2024-04-24T23:14:29.296043+0000","last_became_active":"2024-04-24T23:12:32.024316+0000","last_became_peered":"2024-04-24T23:12:32.024316+0000","last_unstale":"2024-04-24T23:14:29.296043+0000","last_undegraded":"2024-04-24T23:14:29.296043+0000","last_fullsized":"2024-04-24T23:14:29.296043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:21.668782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012232+0000","last_change":"2024-04-24T23:12:27.003435+0000","last_active":"2024-04-24T23:14:29.012232+0000","last_peered":"2024-04-24T23:14:29.012232+0000","last_clean":"2024-04-24T23:14:29.012232+0000","last_became_active":"2024-04-24T23:12:27.003101+0000","last_became_peered":"2024-04-24T23:12:27.003101+0000","last_unstale":"2024-04-24T23:14:29.012232+0000","last_undegraded":"2024-04-24T23:14:29.012232+0000","last_fullsized":"2024-04-24T23:14:29.012232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:42.481545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931602+0000","last_change":"2024-04-24T23:12:28.005056+0000","last_active":"2024-04-24T23:14:28.931602+0000","last_peered":"2024-04-24T23:14:28.931602+0000","last_clean":"2024-04-24T23:14:28.931602+0000","last_became_active":"2024-04-24T23:12:28.004725+0000","last_became_peered":"2024-04-24T23:12:28.004725+0000","last_unstale":"2024-04-24T23:14:28.931602+0000","last_undegraded":"2024-04-24T23:14:28.931602+0000","last_fullsized":"2024-04-24T23:14:28.931602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:49.384950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928478+0000","last_change":"2024-04-24T23:12:32.032947+0000","last_active":"2024-04-24T23:14:28.928478+0000","last_peered":"2024-04-24T23:14:28.928478+0000","last_clean":"2024-04-24T23:14:28.928478+0000","last_became_active":"2024-04-24T23:12:32.032560+0000","last_became_peered":"2024-04-24T23:12:32.032560+0000","last_unstale":"2024-04-24T23:14:28.928478+0000","last_undegraded":"2024-04-24T23:14:28.928478+0000","last_fullsized":"2024-04-24T23:14:28.928478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:29.282938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'6","reported_seq":265,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.034588+0000","last_change":"2024-04-24T23:12:27.000011+0000","last_active":"2024-04-24T23:14:48.034588+0000","last_peered":"2024-04-24T23:14:48.034588+0000","last_clean":"2024-04-24T23:14:48.034588+0000","last_became_active":"2024-04-24T23:12:26.999572+0000","last_became_peered":"2024-04-24T23:12:26.999572+0000","last_unstale":"2024-04-24T23:14:48.034588+0000","last_undegraded":"2024-04-24T23:14:48.034588+0000","last_fullsized":"2024-04-24T23:14:48.034588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:18:50.453964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515416+0000","last_change":"2024-04-24T23:12:28.012808+0000","last_active":"2024-04-24T23:14:29.515416+0000","last_peered":"2024-04-24T23:14:29.515416+0000","last_clean":"2024-04-24T23:14:29.515416+0000","last_became_active":"2024-04-24T23:12:28.012406+0000","last_became_peered":"2024-04-24T23:12:28.012406+0000","last_unstale":"2024-04-24T23:14:29.515416+0000","last_undegraded":"2024-04-24T23:14:29.515416+0000","last_fullsized":"2024-04-24T23:14:29.515416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:19.636582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931289+0000","last_change":"2024-04-24T23:12:32.027156+0000","last_active":"2024-04-24T23:14:28.931289+0000","last_peered":"2024-04-24T23:14:28.931289+0000","last_clean":"2024-04-24T23:14:28.931289+0000","last_became_active":"2024-04-24T23:12:32.026743+0000","last_became_peered":"2024-04-24T23:12:32.026743+0000","last_unstale":"2024-04-24T23:14:28.931289+0000","last_undegraded":"2024-04-24T23:14:28.931289+0000","last_fullsized":"2024-04-24T23:14:28.931289+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:43.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009591+0000","last_change":"2024-04-24T23:12:27.001832+0000","last_active":"2024-04-24T23:14:29.009591+0000","last_peered":"2024-04-24T23:14:29.009591+0000","last_clean":"2024-04-24T23:14:29.009591+0000","last_became_active":"2024-04-24T23:12:27.001482+0000","last_became_peered":"2024-04-24T23:12:27.001482+0000","last_unstale":"2024-04-24T23:14:29.009591+0000","last_undegraded":"2024-04-24T23:14:29.009591+0000","last_fullsized":"2024-04-24T23:14:29.009591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:00:28.949767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296224+0000","last_change":"2024-04-24T23:13:27.960551+0000","last_active":"2024-04-24T23:14:29.296224+0000","last_peered":"2024-04-24T23:14:29.296224+0000","last_clean":"2024-04-24T23:14:29.296224+0000","last_became_active":"2024-04-24T23:13:27.960264+0000","last_became_peered":"2024-04-24T23:13:27.960264+0000","last_unstale":"2024-04-24T23:14:29.296224+0000","last_undegraded":"2024-04-24T23:14:29.296224+0000","last_fullsized":"2024-04-24T23:14:29.296224+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:44:38.758688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008555+0000","last_change":"2024-04-24T23:12:32.023007+0000","last_active":"2024-04-24T23:14:29.008555+0000","last_peered":"2024-04-24T23:14:29.008555+0000","last_clean":"2024-04-24T23:14:29.008555+0000","last_became_active":"2024-04-24T23:12:32.021324+0000","last_became_peered":"2024-04-24T23:12:32.021324+0000","last_unstale":"2024-04-24T23:14:29.008555+0000","last_undegraded":"2024-04-24T23:14:29.008555+0000","last_fullsized":"2024-04-24T23:14:29.008555+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:41.542778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012520+0000","last_change":"2024-04-24T23:12:26.990562+0000","last_active":"2024-04-24T23:14:29.012520+0000","last_peered":"2024-04-24T23:14:29.012520+0000","last_clean":"2024-04-24T23:14:29.012520+0000","last_became_active":"2024-04-24T23:12:26.990146+0000","last_became_peered":"2024-04-24T23:12:26.990146+0000","last_unstale":"2024-04-24T23:14:29.012520+0000","last_undegraded":"2024-04-24T23:14:29.012520+0000","last_fullsized":"2024-04-24T23:14:29.012520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.621051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515294+0000","last_change":"2024-04-24T23:12:28.000954+0000","last_active":"2024-04-24T23:14:29.515294+0000","last_peered":"2024-04-24T23:14:29.515294+0000","last_clean":"2024-04-24T23:14:29.515294+0000","last_became_active":"2024-04-24T23:12:28.000610+0000","last_became_peered":"2024-04-24T23:12:28.000610+0000","last_unstale":"2024-04-24T23:14:29.515294+0000","last_undegraded":"2024-04-24T23:14:29.515294+0000","last_fullsized":"2024-04-24T23:14:29.515294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:32.585919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.516500+0000","last_change":"2024-04-24T23:14:29.516500+0000","last_active":"2024-04-24T23:14:29.516500+0000","last_peered":"2024-04-24T23:14:29.516500+0000","last_clean":"2024-04-24T23:14:29.516500+0000","last_became_active":"2024-04-24T23:14:29.516159+0000","last_became_peered":"2024-04-24T23:14:29.516159+0000","last_unstale":"2024-04-24T23:14:29.516500+0000","last_undegraded":"2024-04-24T23:14:29.516500+0000","last_fullsized":"2024-04-24T23:14:29.516500+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:11.290978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":312,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.043073+0000","last_change":"2024-04-24T23:12:26.996719+0000","last_active":"2024-04-24T23:14:48.043073+0000","last_peered":"2024-04-24T23:14:48.043073+0000","last_clean":"2024-04-24T23:14:48.043073+0000","last_became_active":"2024-04-24T23:12:26.996360+0000","last_became_peered":"2024-04-24T23:12:26.996360+0000","last_unstale":"2024-04-24T23:14:48.043073+0000","last_undegraded":"2024-04-24T23:14:48.043073+0000","last_fullsized":"2024-04-24T23:14:48.043073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:27:53.526902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63913,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":183,"num_write":42,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932164+0000","last_change":"2024-04-24T23:12:28.002889+0000","last_active":"2024-04-24T23:14:28.932164+0000","last_peered":"2024-04-24T23:14:28.932164+0000","last_clean":"2024-04-24T23:14:28.932164+0000","last_became_active":"2024-04-24T23:12:28.002540+0000","last_became_peered":"2024-04-24T23:12:28.002540+0000","last_unstale":"2024-04-24T23:14:28.932164+0000","last_undegraded":"2024-04-24T23:14:28.932164+0000","last_fullsized":"2024-04-24T23:14:28.932164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:26:57.792628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012875+0000","last_change":"2024-04-24T23:12:32.031160+0000","last_active":"2024-04-24T23:14:29.012875+0000","last_peered":"2024-04-24T23:14:29.012875+0000","last_clean":"2024-04-24T23:14:29.012875+0000","last_became_active":"2024-04-24T23:12:32.030719+0000","last_became_peered":"2024-04-24T23:12:32.030719+0000","last_unstale":"2024-04-24T23:14:29.012875+0000","last_undegraded":"2024-04-24T23:14:29.012875+0000","last_fullsized":"2024-04-24T23:14:29.012875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:45.648011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515047+0000","last_change":"2024-04-24T23:13:27.967294+0000","last_active":"2024-04-24T23:14:29.515047+0000","last_peered":"2024-04-24T23:14:29.515047+0000","last_clean":"2024-04-24T23:14:29.515047+0000","last_became_active":"2024-04-24T23:13:27.958064+0000","last_became_peered":"2024-04-24T23:13:27.958064+0000","last_unstale":"2024-04-24T23:14:29.515047+0000","last_undegraded":"2024-04-24T23:14:29.515047+0000","last_fullsized":"2024-04-24T23:14:29.515047+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:30:08.824502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931284+0000","last_change":"2024-04-24T23:12:28.003211+0000","last_active":"2024-04-24T23:14:28.931284+0000","last_peered":"2024-04-24T23:14:28.931284+0000","last_clean":"2024-04-24T23:14:28.931284+0000","last_became_active":"2024-04-24T23:12:28.002897+0000","last_became_peered":"2024-04-24T23:12:28.002897+0000","last_unstale":"2024-04-24T23:14:28.931284+0000","last_undegraded":"2024-04-24T23:14:28.931284+0000","last_fullsized":"2024-04-24T23:14:28.931284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.187239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515101+0000","last_change":"2024-04-24T23:12:32.028857+0000","last_active":"2024-04-24T23:14:29.515101+0000","last_peered":"2024-04-24T23:14:29.515101+0000","last_clean":"2024-04-24T23:14:29.515101+0000","last_became_active":"2024-04-24T23:12:32.028455+0000","last_became_peered":"2024-04-24T23:12:32.028455+0000","last_unstale":"2024-04-24T23:14:29.515101+0000","last_undegraded":"2024-04-24T23:14:29.515101+0000","last_fullsized":"2024-04-24T23:14:29.515101+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:23.073895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012600+0000","last_change":"2024-04-24T23:12:27.003525+0000","last_active":"2024-04-24T23:14:29.012600+0000","last_peered":"2024-04-24T23:14:29.012600+0000","last_clean":"2024-04-24T23:14:29.012600+0000","last_became_active":"2024-04-24T23:12:27.003203+0000","last_became_peered":"2024-04-24T23:12:27.003203+0000","last_unstale":"2024-04-24T23:14:29.012600+0000","last_undegraded":"2024-04-24T23:14:29.012600+0000","last_fullsized":"2024-04-24T23:14:29.012600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:47:18.521925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009356+0000","last_change":"2024-04-24T23:12:28.017111+0000","last_active":"2024-04-24T23:14:29.009356+0000","last_peered":"2024-04-24T23:14:29.009356+0000","last_clean":"2024-04-24T23:14:29.009356+0000","last_became_active":"2024-04-24T23:12:28.016725+0000","last_became_peered":"2024-04-24T23:12:28.016725+0000","last_unstale":"2024-04-24T23:14:29.009356+0000","last_undegraded":"2024-04-24T23:14:29.009356+0000","last_fullsized":"2024-04-24T23:14:29.009356+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:25.579296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.297405+0000","last_change":"2024-04-24T23:14:29.297405+0000","last_active":"2024-04-24T23:14:29.297405+0000","last_peered":"2024-04-24T23:14:29.297405+0000","last_clean":"2024-04-24T23:14:29.297405+0000","last_became_active":"2024-04-24T23:14:29.296936+0000","last_became_peered":"2024-04-24T23:14:29.296936+0000","last_unstale":"2024-04-24T23:14:29.297405+0000","last_undegraded":"2024-04-24T23:14:29.297405+0000","last_fullsized":"2024-04-24T23:14:29.297405+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:53.458679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,7,0,3],"acting":[10,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":156,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:50.035879+0000","last_change":"2024-04-24T23:12:27.002797+0000","last_active":"2024-04-24T23:14:50.035879+0000","last_peered":"2024-04-24T23:14:50.035879+0000","last_clean":"2024-04-24T23:14:50.035879+0000","last_became_active":"2024-04-24T23:12:27.002086+0000","last_became_peered":"2024-04-24T23:12:27.002086+0000","last_unstale":"2024-04-24T23:14:50.035879+0000","last_undegraded":"2024-04-24T23:14:50.035879+0000","last_fullsized":"2024-04-24T23:14:50.035879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:39:55.025453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009610+0000","last_change":"2024-04-24T23:13:27.955130+0000","last_active":"2024-04-24T23:14:29.009610+0000","last_peered":"2024-04-24T23:14:29.009610+0000","last_clean":"2024-04-24T23:14:29.009610+0000","last_became_active":"2024-04-24T23:13:27.954784+0000","last_became_peered":"2024-04-24T23:13:27.954784+0000","last_unstale":"2024-04-24T23:14:29.009610+0000","last_undegraded":"2024-04-24T23:14:29.009610+0000","last_fullsized":"2024-04-24T23:14:29.009610+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:10.574771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931931+0000","last_change":"2024-04-24T23:12:32.029104+0000","last_active":"2024-04-24T23:14:28.931931+0000","last_peered":"2024-04-24T23:14:28.931931+0000","last_clean":"2024-04-24T23:14:28.931931+0000","last_became_active":"2024-04-24T23:12:32.028704+0000","last_became_peered":"2024-04-24T23:12:32.028704+0000","last_unstale":"2024-04-24T23:14:28.931931+0000","last_undegraded":"2024-04-24T23:14:28.931931+0000","last_fullsized":"2024-04-24T23:14:28.931931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:20:33.200884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.514991+0000","last_change":"2024-04-24T23:13:27.976835+0000","last_active":"2024-04-24T23:14:29.514991+0000","last_peered":"2024-04-24T23:14:29.514991+0000","last_clean":"2024-04-24T23:14:29.514991+0000","last_became_active":"2024-04-24T23:13:27.959684+0000","last_became_peered":"2024-04-24T23:13:27.959684+0000","last_unstale":"2024-04-24T23:14:29.514991+0000","last_undegraded":"2024-04-24T23:14:29.514991+0000","last_fullsized":"2024-04-24T23:14:29.514991+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:24.327964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012405+0000","last_change":"2024-04-24T23:12:28.010510+0000","last_active":"2024-04-24T23:14:29.012405+0000","last_peered":"2024-04-24T23:14:29.012405+0000","last_clean":"2024-04-24T23:14:29.012405+0000","last_became_active":"2024-04-24T23:12:28.009244+0000","last_became_peered":"2024-04-24T23:12:28.009244+0000","last_unstale":"2024-04-24T23:14:29.012405+0000","last_undegraded":"2024-04-24T23:14:29.012405+0000","last_fullsized":"2024-04-24T23:14:29.012405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:52:39.146563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012453+0000","last_change":"2024-04-24T23:12:32.031156+0000","last_active":"2024-04-24T23:14:29.012453+0000","last_peered":"2024-04-24T23:14:29.012453+0000","last_clean":"2024-04-24T23:14:29.012453+0000","last_became_active":"2024-04-24T23:12:32.030653+0000","last_became_peered":"2024-04-24T23:12:32.030653+0000","last_unstale":"2024-04-24T23:14:29.012453+0000","last_undegraded":"2024-04-24T23:14:29.012453+0000","last_fullsized":"2024-04-24T23:14:29.012453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:38:48.369239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932509+0000","last_change":"2024-04-24T23:12:26.998003+0000","last_active":"2024-04-24T23:14:28.932509+0000","last_peered":"2024-04-24T23:14:28.932509+0000","last_clean":"2024-04-24T23:14:28.932509+0000","last_became_active":"2024-04-24T23:12:26.997661+0000","last_became_peered":"2024-04-24T23:12:26.997661+0000","last_unstale":"2024-04-24T23:14:28.932509+0000","last_undegraded":"2024-04-24T23:14:28.932509+0000","last_fullsized":"2024-04-24T23:14:28.932509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:17.995397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515371+0000","last_change":"2024-04-24T23:12:28.004822+0000","last_active":"2024-04-24T23:14:29.515371+0000","last_peered":"2024-04-24T23:14:29.515371+0000","last_clean":"2024-04-24T23:14:29.515371+0000","last_became_active":"2024-04-24T23:12:28.004334+0000","last_became_peered":"2024-04-24T23:12:28.004334+0000","last_unstale":"2024-04-24T23:14:29.515371+0000","last_undegraded":"2024-04-24T23:14:29.515371+0000","last_fullsized":"2024-04-24T23:14:29.515371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:26:30.717796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931518+0000","last_change":"2024-04-24T23:12:32.085309+0000","last_active":"2024-04-24T23:14:28.931518+0000","last_peered":"2024-04-24T23:14:28.931518+0000","last_clean":"2024-04-24T23:14:28.931518+0000","last_became_active":"2024-04-24T23:12:32.084892+0000","last_became_peered":"2024-04-24T23:12:32.084892+0000","last_unstale":"2024-04-24T23:14:28.931518+0000","last_undegraded":"2024-04-24T23:14:28.931518+0000","last_fullsized":"2024-04-24T23:14:28.931518+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:20.318901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932253+0000","last_change":"2024-04-24T23:12:27.005096+0000","last_active":"2024-04-24T23:14:28.932253+0000","last_peered":"2024-04-24T23:14:28.932253+0000","last_clean":"2024-04-24T23:14:28.932253+0000","last_became_active":"2024-04-24T23:12:27.004699+0000","last_became_peered":"2024-04-24T23:12:27.004699+0000","last_unstale":"2024-04-24T23:14:28.932253+0000","last_undegraded":"2024-04-24T23:14:28.932253+0000","last_fullsized":"2024-04-24T23:14:28.932253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:58.225361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931600+0000","last_change":"2024-04-24T23:12:28.012230+0000","last_active":"2024-04-24T23:14:28.931600+0000","last_peered":"2024-04-24T23:14:28.931600+0000","last_clean":"2024-04-24T23:14:28.931600+0000","last_became_active":"2024-04-24T23:12:28.011900+0000","last_became_peered":"2024-04-24T23:12:28.011900+0000","last_unstale":"2024-04-24T23:14:28.931600+0000","last_undegraded":"2024-04-24T23:14:28.931600+0000","last_fullsized":"2024-04-24T23:14:28.931600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.780548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932162+0000","last_change":"2024-04-24T23:12:32.026388+0000","last_active":"2024-04-24T23:14:28.932162+0000","last_peered":"2024-04-24T23:14:28.932162+0000","last_clean":"2024-04-24T23:14:28.932162+0000","last_became_active":"2024-04-24T23:12:32.025994+0000","last_became_peered":"2024-04-24T23:12:32.025994+0000","last_unstale":"2024-04-24T23:14:28.932162+0000","last_undegraded":"2024-04-24T23:14:28.932162+0000","last_fullsized":"2024-04-24T23:14:28.932162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:13.062781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":153,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.034564+0000","last_change":"2024-04-24T23:12:27.005080+0000","last_active":"2024-04-24T23:14:49.034564+0000","last_peered":"2024-04-24T23:14:49.034564+0000","last_clean":"2024-04-24T23:14:49.034564+0000","last_became_active":"2024-04-24T23:12:27.004740+0000","last_became_peered":"2024-04-24T23:12:27.004740+0000","last_unstale":"2024-04-24T23:14:49.034564+0000","last_undegraded":"2024-04-24T23:14:49.034564+0000","last_fullsized":"2024-04-24T23:14:49.034564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:09:22.502399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931619+0000","last_change":"2024-04-24T23:12:28.008016+0000","last_active":"2024-04-24T23:14:28.931619+0000","last_peered":"2024-04-24T23:14:28.931619+0000","last_clean":"2024-04-24T23:14:28.931619+0000","last_became_active":"2024-04-24T23:12:28.007139+0000","last_became_peered":"2024-04-24T23:12:28.007139+0000","last_unstale":"2024-04-24T23:14:28.931619+0000","last_undegraded":"2024-04-24T23:14:28.931619+0000","last_fullsized":"2024-04-24T23:14:28.931619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:01.698193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932114+0000","last_change":"2024-04-24T23:12:32.085290+0000","last_active":"2024-04-24T23:14:28.932114+0000","last_peered":"2024-04-24T23:14:28.932114+0000","last_clean":"2024-04-24T23:14:28.932114+0000","last_became_active":"2024-04-24T23:12:32.084852+0000","last_became_peered":"2024-04-24T23:12:32.084852+0000","last_unstale":"2024-04-24T23:14:28.932114+0000","last_undegraded":"2024-04-24T23:14:28.932114+0000","last_fullsized":"2024-04-24T23:14:28.932114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:36.753633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012753+0000","last_change":"2024-04-24T23:12:28.007552+0000","last_active":"2024-04-24T23:14:29.012753+0000","last_peered":"2024-04-24T23:14:29.012753+0000","last_clean":"2024-04-24T23:14:29.012753+0000","last_became_active":"2024-04-24T23:12:28.007196+0000","last_became_peered":"2024-04-24T23:12:28.007196+0000","last_unstale":"2024-04-24T23:14:29.012753+0000","last_undegraded":"2024-04-24T23:14:29.012753+0000","last_fullsized":"2024-04-24T23:14:29.012753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:15.695485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:48.032904+0000","last_change":"2024-04-24T23:12:26.990814+0000","last_active":"2024-04-24T23:14:48.032904+0000","last_peered":"2024-04-24T23:14:48.032904+0000","last_clean":"2024-04-24T23:14:48.032904+0000","last_became_active":"2024-04-24T23:12:26.990476+0000","last_became_peered":"2024-04-24T23:12:26.990476+0000","last_unstale":"2024-04-24T23:14:48.032904+0000","last_undegraded":"2024-04-24T23:14:48.032904+0000","last_fullsized":"2024-04-24T23:14:48.032904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:49:32.645770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009649+0000","last_change":"2024-04-24T23:12:32.022039+0000","last_active":"2024-04-24T23:14:29.009649+0000","last_peered":"2024-04-24T23:14:29.009649+0000","last_clean":"2024-04-24T23:14:29.009649+0000","last_became_active":"2024-04-24T23:12:32.021514+0000","last_became_peered":"2024-04-24T23:12:32.021514+0000","last_unstale":"2024-04-24T23:14:29.009649+0000","last_undegraded":"2024-04-24T23:14:29.009649+0000","last_fullsized":"2024-04-24T23:14:29.009649+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:42.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928756+0000","last_change":"2024-04-24T23:12:28.008676+0000","last_active":"2024-04-24T23:14:28.928756+0000","last_peered":"2024-04-24T23:14:28.928756+0000","last_clean":"2024-04-24T23:14:28.928756+0000","last_became_active":"2024-04-24T23:12:28.008376+0000","last_became_peered":"2024-04-24T23:12:28.008376+0000","last_unstale":"2024-04-24T23:14:28.928756+0000","last_undegraded":"2024-04-24T23:14:28.928756+0000","last_fullsized":"2024-04-24T23:14:28.928756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:01:15.358332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008483+0000","last_change":"2024-04-24T23:12:26.994952+0000","last_active":"2024-04-24T23:14:29.008483+0000","last_peered":"2024-04-24T23:14:29.008483+0000","last_clean":"2024-04-24T23:14:29.008483+0000","last_became_active":"2024-04-24T23:12:26.994556+0000","last_became_peered":"2024-04-24T23:12:26.994556+0000","last_unstale":"2024-04-24T23:14:29.008483+0000","last_undegraded":"2024-04-24T23:14:29.008483+0000","last_fullsized":"2024-04-24T23:14:29.008483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:14.257427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":354,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009006+0000","last_change":"2024-04-24T23:11:16.721876+0000","last_active":"2024-04-24T23:14:29.009006+0000","last_peered":"2024-04-24T23:14:29.009006+0000","last_clean":"2024-04-24T23:14:29.009006+0000","last_became_active":"2024-04-24T23:11:16.721291+0000","last_became_peered":"2024-04-24T23:11:16.721291+0000","last_unstale":"2024-04-24T23:14:29.009006+0000","last_undegraded":"2024-04-24T23:14:29.009006+0000","last_fullsized":"2024-04-24T23:14:29.009006+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:07:51.257753+0000","last_clean_scrub_stamp":"2024-04-24T23:07:51.257753+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:28.458939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.013052+0000","last_change":"2024-04-24T23:12:32.085905+0000","last_active":"2024-04-24T23:14:29.013052+0000","last_peered":"2024-04-24T23:14:29.013052+0000","last_clean":"2024-04-24T23:14:29.013052+0000","last_became_active":"2024-04-24T23:12:32.085374+0000","last_became_peered":"2024-04-24T23:12:32.085374+0000","last_unstale":"2024-04-24T23:14:29.013052+0000","last_undegraded":"2024-04-24T23:14:29.013052+0000","last_fullsized":"2024-04-24T23:14:29.013052+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:16.290651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931162+0000","last_change":"2024-04-24T23:12:27.999880+0000","last_active":"2024-04-24T23:14:28.931162+0000","last_peered":"2024-04-24T23:14:28.931162+0000","last_clean":"2024-04-24T23:14:28.931162+0000","last_became_active":"2024-04-24T23:12:27.999543+0000","last_became_peered":"2024-04-24T23:12:27.999543+0000","last_unstale":"2024-04-24T23:14:28.931162+0000","last_undegraded":"2024-04-24T23:14:28.931162+0000","last_fullsized":"2024-04-24T23:14:28.931162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:44.758968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009273+0000","last_change":"2024-04-24T23:13:28.341009+0000","last_active":"2024-04-24T23:14:29.009273+0000","last_peered":"2024-04-24T23:14:29.009273+0000","last_clean":"2024-04-24T23:14:29.009273+0000","last_became_active":"2024-04-24T23:13:28.338192+0000","last_became_peered":"2024-04-24T23:13:28.338192+0000","last_unstale":"2024-04-24T23:14:29.009273+0000","last_undegraded":"2024-04-24T23:14:29.009273+0000","last_fullsized":"2024-04-24T23:14:29.009273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:11.187364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515225+0000","last_change":"2024-04-24T23:12:32.024952+0000","last_active":"2024-04-24T23:14:29.515225+0000","last_peered":"2024-04-24T23:14:29.515225+0000","last_clean":"2024-04-24T23:14:29.515225+0000","last_became_active":"2024-04-24T23:12:32.024404+0000","last_became_peered":"2024-04-24T23:12:32.024404+0000","last_unstale":"2024-04-24T23:14:29.515225+0000","last_undegraded":"2024-04-24T23:14:29.515225+0000","last_fullsized":"2024-04-24T23:14:29.515225+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:07:58.377794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932193+0000","last_change":"2024-04-24T23:12:27.007191+0000","last_active":"2024-04-24T23:14:28.932193+0000","last_peered":"2024-04-24T23:14:28.932193+0000","last_clean":"2024-04-24T23:14:28.932193+0000","last_became_active":"2024-04-24T23:12:27.006867+0000","last_became_peered":"2024-04-24T23:12:27.006867+0000","last_unstale":"2024-04-24T23:14:28.932193+0000","last_undegraded":"2024-04-24T23:14:28.932193+0000","last_fullsized":"2024-04-24T23:14:28.932193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:35:58.112293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932401+0000","last_change":"2024-04-24T23:12:27.996322+0000","last_active":"2024-04-24T23:14:28.932401+0000","last_peered":"2024-04-24T23:14:28.932401+0000","last_clean":"2024-04-24T23:14:28.932401+0000","last_became_active":"2024-04-24T23:12:27.995991+0000","last_became_peered":"2024-04-24T23:12:27.995991+0000","last_unstale":"2024-04-24T23:14:28.932401+0000","last_undegraded":"2024-04-24T23:14:28.932401+0000","last_fullsized":"2024-04-24T23:14:28.932401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:17:08.960347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928414+0000","last_change":"2024-04-24T23:12:32.084959+0000","last_active":"2024-04-24T23:14:28.928414+0000","last_peered":"2024-04-24T23:14:28.928414+0000","last_clean":"2024-04-24T23:14:28.928414+0000","last_became_active":"2024-04-24T23:12:32.084553+0000","last_became_peered":"2024-04-24T23:12:32.084553+0000","last_unstale":"2024-04-24T23:14:28.928414+0000","last_undegraded":"2024-04-24T23:14:28.928414+0000","last_fullsized":"2024-04-24T23:14:28.928414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:35.836337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932585+0000","last_change":"2024-04-24T23:12:27.007742+0000","last_active":"2024-04-24T23:14:28.932585+0000","last_peered":"2024-04-24T23:14:28.932585+0000","last_clean":"2024-04-24T23:14:28.932585+0000","last_became_active":"2024-04-24T23:12:27.007423+0000","last_became_peered":"2024-04-24T23:12:27.007423+0000","last_unstale":"2024-04-24T23:14:28.932585+0000","last_undegraded":"2024-04-24T23:14:28.932585+0000","last_fullsized":"2024-04-24T23:14:28.932585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:30:39.478356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928898+0000","last_change":"2024-04-24T23:12:28.010961+0000","last_active":"2024-04-24T23:14:28.928898+0000","last_peered":"2024-04-24T23:14:28.928898+0000","last_clean":"2024-04-24T23:14:28.928898+0000","last_became_active":"2024-04-24T23:12:28.010641+0000","last_became_peered":"2024-04-24T23:12:28.010641+0000","last_unstale":"2024-04-24T23:14:28.928898+0000","last_undegraded":"2024-04-24T23:14:28.928898+0000","last_fullsized":"2024-04-24T23:14:28.928898+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:33:43.555358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929416+0000","last_change":"2024-04-24T23:12:32.023687+0000","last_active":"2024-04-24T23:14:28.929416+0000","last_peered":"2024-04-24T23:14:28.929416+0000","last_clean":"2024-04-24T23:14:28.929416+0000","last_became_active":"2024-04-24T23:12:32.023052+0000","last_became_peered":"2024-04-24T23:12:32.023052+0000","last_unstale":"2024-04-24T23:14:28.929416+0000","last_undegraded":"2024-04-24T23:14:28.929416+0000","last_fullsized":"2024-04-24T23:14:28.929416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:39.854030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008925+0000","last_change":"2024-04-24T23:12:27.009743+0000","last_active":"2024-04-24T23:14:29.008925+0000","last_peered":"2024-04-24T23:14:29.008925+0000","last_clean":"2024-04-24T23:14:29.008925+0000","last_became_active":"2024-04-24T23:12:27.009343+0000","last_became_peered":"2024-04-24T23:12:27.009343+0000","last_unstale":"2024-04-24T23:14:29.008925+0000","last_undegraded":"2024-04-24T23:14:29.008925+0000","last_fullsized":"2024-04-24T23:14:29.008925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:20.387738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.296261+0000","last_change":"2024-04-24T23:13:27.961769+0000","last_active":"2024-04-24T23:14:29.296261+0000","last_peered":"2024-04-24T23:14:29.296261+0000","last_clean":"2024-04-24T23:14:29.296261+0000","last_became_active":"2024-04-24T23:13:27.961460+0000","last_became_peered":"2024-04-24T23:13:27.961460+0000","last_unstale":"2024-04-24T23:14:29.296261+0000","last_undegraded":"2024-04-24T23:14:29.296261+0000","last_fullsized":"2024-04-24T23:14:29.296261+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:01.012688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009238+0000","last_change":"2024-04-24T23:12:32.086209+0000","last_active":"2024-04-24T23:14:29.009238+0000","last_peered":"2024-04-24T23:14:29.009238+0000","last_clean":"2024-04-24T23:14:29.009238+0000","last_became_active":"2024-04-24T23:12:32.085752+0000","last_became_peered":"2024-04-24T23:12:32.085752+0000","last_unstale":"2024-04-24T23:14:29.009238+0000","last_undegraded":"2024-04-24T23:14:29.009238+0000","last_fullsized":"2024-04-24T23:14:29.009238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:53:52.120905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515056+0000","last_change":"2024-04-24T23:12:26.996355+0000","last_active":"2024-04-24T23:14:29.515056+0000","last_peered":"2024-04-24T23:14:29.515056+0000","last_clean":"2024-04-24T23:14:29.515056+0000","last_became_active":"2024-04-24T23:12:26.995486+0000","last_became_peered":"2024-04-24T23:12:26.995486+0000","last_unstale":"2024-04-24T23:14:29.515056+0000","last_undegraded":"2024-04-24T23:14:29.515056+0000","last_fullsized":"2024-04-24T23:14:29.515056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:04:48.981985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931625+0000","last_change":"2024-04-24T23:12:28.004610+0000","last_active":"2024-04-24T23:14:28.931625+0000","last_peered":"2024-04-24T23:14:28.931625+0000","last_clean":"2024-04-24T23:14:28.931625+0000","last_became_active":"2024-04-24T23:12:28.004051+0000","last_became_peered":"2024-04-24T23:12:28.004051+0000","last_unstale":"2024-04-24T23:14:28.931625+0000","last_undegraded":"2024-04-24T23:14:28.931625+0000","last_fullsized":"2024-04-24T23:14:28.931625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:45:42.656954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.014680+0000","last_change":"2024-04-24T23:14:29.014103+0000","last_active":"2024-04-24T23:14:29.014680+0000","last_peered":"2024-04-24T23:14:29.014680+0000","last_clean":"2024-04-24T23:14:29.014680+0000","last_became_active":"2024-04-24T23:14:29.013472+0000","last_became_peered":"2024-04-24T23:14:29.013472+0000","last_unstale":"2024-04-24T23:14:29.014680+0000","last_undegraded":"2024-04-24T23:14:29.014680+0000","last_fullsized":"2024-04-24T23:14:29.014680+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:37.823011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928576+0000","last_change":"2024-04-24T23:12:27.006357+0000","last_active":"2024-04-24T23:14:28.928576+0000","last_peered":"2024-04-24T23:14:28.928576+0000","last_clean":"2024-04-24T23:14:28.928576+0000","last_became_active":"2024-04-24T23:12:27.006040+0000","last_became_peered":"2024-04-24T23:12:27.006040+0000","last_unstale":"2024-04-24T23:14:28.928576+0000","last_undegraded":"2024-04-24T23:14:28.928576+0000","last_fullsized":"2024-04-24T23:14:28.928576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:27.520663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515601+0000","last_change":"2024-04-24T23:12:28.008909+0000","last_active":"2024-04-24T23:14:29.515601+0000","last_peered":"2024-04-24T23:14:29.515601+0000","last_clean":"2024-04-24T23:14:29.515601+0000","last_became_active":"2024-04-24T23:12:28.008552+0000","last_became_peered":"2024-04-24T23:12:28.008552+0000","last_unstale":"2024-04-24T23:14:29.515601+0000","last_undegraded":"2024-04-24T23:14:29.515601+0000","last_fullsized":"2024-04-24T23:14:29.515601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:20.628234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008834+0000","last_change":"2024-04-24T23:12:32.029720+0000","last_active":"2024-04-24T23:14:29.008834+0000","last_peered":"2024-04-24T23:14:29.008834+0000","last_clean":"2024-04-24T23:14:29.008834+0000","last_became_active":"2024-04-24T23:12:32.029311+0000","last_became_peered":"2024-04-24T23:12:32.029311+0000","last_unstale":"2024-04-24T23:14:29.008834+0000","last_undegraded":"2024-04-24T23:14:29.008834+0000","last_fullsized":"2024-04-24T23:14:29.008834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:37.832102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:40.286863+0000","last_change":"2024-04-24T23:12:26.995552+0000","last_active":"2024-04-24T23:14:40.286863+0000","last_peered":"2024-04-24T23:14:40.286863+0000","last_clean":"2024-04-24T23:14:40.286863+0000","last_became_active":"2024-04-24T23:12:26.994922+0000","last_became_peered":"2024-04-24T23:12:26.994922+0000","last_unstale":"2024-04-24T23:14:40.286863+0000","last_undegraded":"2024-04-24T23:14:40.286863+0000","last_fullsized":"2024-04-24T23:14:40.286863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:11:17.981344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928695+0000","last_change":"2024-04-24T23:12:28.008802+0000","last_active":"2024-04-24T23:14:28.928695+0000","last_peered":"2024-04-24T23:14:28.928695+0000","last_clean":"2024-04-24T23:14:28.928695+0000","last_became_active":"2024-04-24T23:12:28.008496+0000","last_became_peered":"2024-04-24T23:12:28.008496+0000","last_unstale":"2024-04-24T23:14:28.928695+0000","last_undegraded":"2024-04-24T23:14:28.928695+0000","last_fullsized":"2024-04-24T23:14:28.928695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:40:54.638298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.008767+0000","last_change":"2024-04-24T23:12:32.085555+0000","last_active":"2024-04-24T23:14:29.008767+0000","last_peered":"2024-04-24T23:14:29.008767+0000","last_clean":"2024-04-24T23:14:29.008767+0000","last_became_active":"2024-04-24T23:12:32.085085+0000","last_became_peered":"2024-04-24T23:12:32.085085+0000","last_unstale":"2024-04-24T23:14:29.008767+0000","last_undegraded":"2024-04-24T23:14:29.008767+0000","last_fullsized":"2024-04-24T23:14:29.008767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:24.563964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932112+0000","last_change":"2024-04-24T23:13:28.342962+0000","last_active":"2024-04-24T23:14:28.932112+0000","last_peered":"2024-04-24T23:14:28.932112+0000","last_clean":"2024-04-24T23:14:28.932112+0000","last_became_active":"2024-04-24T23:13:28.339742+0000","last_became_peered":"2024-04-24T23:13:28.339742+0000","last_unstale":"2024-04-24T23:14:28.932112+0000","last_undegraded":"2024-04-24T23:14:28.932112+0000","last_fullsized":"2024-04-24T23:14:28.932112+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:38:36.910863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009444+0000","last_change":"2024-04-24T23:12:28.014865+0000","last_active":"2024-04-24T23:14:29.009444+0000","last_peered":"2024-04-24T23:14:29.009444+0000","last_clean":"2024-04-24T23:14:29.009444+0000","last_became_active":"2024-04-24T23:12:28.014538+0000","last_became_peered":"2024-04-24T23:12:28.014538+0000","last_unstale":"2024-04-24T23:14:29.009444+0000","last_undegraded":"2024-04-24T23:14:29.009444+0000","last_fullsized":"2024-04-24T23:14:29.009444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:22:55.306883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515116+0000","last_change":"2024-04-24T23:12:32.024849+0000","last_active":"2024-04-24T23:14:29.515116+0000","last_peered":"2024-04-24T23:14:29.515116+0000","last_clean":"2024-04-24T23:14:29.515116+0000","last_became_active":"2024-04-24T23:12:32.024412+0000","last_became_peered":"2024-04-24T23:12:32.024412+0000","last_unstale":"2024-04-24T23:14:29.515116+0000","last_undegraded":"2024-04-24T23:14:29.515116+0000","last_fullsized":"2024-04-24T23:14:29.515116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:34:16.884565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:49.036058+0000","last_change":"2024-04-24T23:12:27.007284+0000","last_active":"2024-04-24T23:14:49.036058+0000","last_peered":"2024-04-24T23:14:49.036058+0000","last_clean":"2024-04-24T23:14:49.036058+0000","last_became_active":"2024-04-24T23:12:27.006916+0000","last_became_peered":"2024-04-24T23:12:27.006916+0000","last_unstale":"2024-04-24T23:14:49.036058+0000","last_undegraded":"2024-04-24T23:14:49.036058+0000","last_fullsized":"2024-04-24T23:14:49.036058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:11:40.333387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.010029+0000","last_change":"2024-04-24T23:12:28.011603+0000","last_active":"2024-04-24T23:14:29.010029+0000","last_peered":"2024-04-24T23:14:29.010029+0000","last_clean":"2024-04-24T23:14:29.010029+0000","last_became_active":"2024-04-24T23:12:28.011268+0000","last_became_peered":"2024-04-24T23:12:28.011268+0000","last_unstale":"2024-04-24T23:14:29.010029+0000","last_undegraded":"2024-04-24T23:14:29.010029+0000","last_fullsized":"2024-04-24T23:14:29.010029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.693024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295803+0000","last_change":"2024-04-24T23:12:32.033965+0000","last_active":"2024-04-24T23:14:29.295803+0000","last_peered":"2024-04-24T23:14:29.295803+0000","last_clean":"2024-04-24T23:14:29.295803+0000","last_became_active":"2024-04-24T23:12:32.033472+0000","last_became_peered":"2024-04-24T23:12:32.033472+0000","last_unstale":"2024-04-24T23:14:29.295803+0000","last_undegraded":"2024-04-24T23:14:29.295803+0000","last_fullsized":"2024-04-24T23:14:29.295803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:05.243047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012764+0000","last_change":"2024-04-24T23:12:26.991981+0000","last_active":"2024-04-24T23:14:29.012764+0000","last_peered":"2024-04-24T23:14:29.012764+0000","last_clean":"2024-04-24T23:14:29.012764+0000","last_became_active":"2024-04-24T23:12:26.990707+0000","last_became_peered":"2024-04-24T23:12:26.990707+0000","last_unstale":"2024-04-24T23:14:29.012764+0000","last_undegraded":"2024-04-24T23:14:29.012764+0000","last_fullsized":"2024-04-24T23:14:29.012764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:36.317443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009154+0000","last_change":"2024-04-24T23:12:28.010428+0000","last_active":"2024-04-24T23:14:29.009154+0000","last_peered":"2024-04-24T23:14:29.009154+0000","last_clean":"2024-04-24T23:14:29.009154+0000","last_became_active":"2024-04-24T23:12:28.010096+0000","last_became_peered":"2024-04-24T23:12:28.010096+0000","last_unstale":"2024-04-24T23:14:29.009154+0000","last_undegraded":"2024-04-24T23:14:29.009154+0000","last_fullsized":"2024-04-24T23:14:29.009154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:31.949617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012035+0000","last_change":"2024-04-24T23:12:32.023828+0000","last_active":"2024-04-24T23:14:29.012035+0000","last_peered":"2024-04-24T23:14:29.012035+0000","last_clean":"2024-04-24T23:14:29.012035+0000","last_became_active":"2024-04-24T23:12:32.023437+0000","last_became_peered":"2024-04-24T23:12:32.023437+0000","last_unstale":"2024-04-24T23:14:29.012035+0000","last_undegraded":"2024-04-24T23:14:29.012035+0000","last_fullsized":"2024-04-24T23:14:29.012035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:31.002915+0000","last_clean_scrub_stamp":"2024-04-24T23:12:31.002915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.186860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931862+0000","last_change":"2024-04-24T23:12:26.993711+0000","last_active":"2024-04-24T23:14:28.931862+0000","last_peered":"2024-04-24T23:14:28.931862+0000","last_clean":"2024-04-24T23:14:28.931862+0000","last_became_active":"2024-04-24T23:12:26.993368+0000","last_became_peered":"2024-04-24T23:12:26.993368+0000","last_unstale":"2024-04-24T23:14:28.931862+0000","last_undegraded":"2024-04-24T23:14:28.931862+0000","last_fullsized":"2024-04-24T23:14:28.931862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:25.970339+0000","last_clean_scrub_stamp":"2024-04-24T23:12:25.970339+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:58.843747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.009225+0000","last_change":"2024-04-24T23:12:28.003026+0000","last_active":"2024-04-24T23:14:29.009225+0000","last_peered":"2024-04-24T23:14:29.009225+0000","last_clean":"2024-04-24T23:14:29.009225+0000","last_became_active":"2024-04-24T23:12:28.002693+0000","last_became_peered":"2024-04-24T23:12:28.002693+0000","last_unstale":"2024-04-24T23:14:29.009225+0000","last_undegraded":"2024-04-24T23:14:29.009225+0000","last_fullsized":"2024-04-24T23:14:29.009225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:24:30.926251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515390+0000","last_change":"2024-04-24T23:12:28.013414+0000","last_active":"2024-04-24T23:14:29.515390+0000","last_peered":"2024-04-24T23:14:29.515390+0000","last_clean":"2024-04-24T23:14:29.515390+0000","last_became_active":"2024-04-24T23:12:28.013030+0000","last_became_peered":"2024-04-24T23:12:28.013030+0000","last_unstale":"2024-04-24T23:14:29.515390+0000","last_undegraded":"2024-04-24T23:14:29.515390+0000","last_fullsized":"2024-04-24T23:14:29.515390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:35:48.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930086+0000","last_change":"2024-04-24T23:12:28.004821+0000","last_active":"2024-04-24T23:14:28.930086+0000","last_peered":"2024-04-24T23:14:28.930086+0000","last_clean":"2024-04-24T23:14:28.930086+0000","last_became_active":"2024-04-24T23:12:28.003162+0000","last_became_peered":"2024-04-24T23:12:28.003162+0000","last_unstale":"2024-04-24T23:14:28.930086+0000","last_undegraded":"2024-04-24T23:14:28.930086+0000","last_fullsized":"2024-04-24T23:14:28.930086+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:59.109665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.929419+0000","last_change":"2024-04-24T23:12:28.002115+0000","last_active":"2024-04-24T23:14:28.929419+0000","last_peered":"2024-04-24T23:14:28.929419+0000","last_clean":"2024-04-24T23:14:28.929419+0000","last_became_active":"2024-04-24T23:12:28.001727+0000","last_became_peered":"2024-04-24T23:12:28.001727+0000","last_unstale":"2024-04-24T23:14:28.929419+0000","last_undegraded":"2024-04-24T23:14:28.929419+0000","last_fullsized":"2024-04-24T23:14:28.929419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:49:14.348843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.295994+0000","last_change":"2024-04-24T23:12:28.007784+0000","last_active":"2024-04-24T23:14:29.295994+0000","last_peered":"2024-04-24T23:14:29.295994+0000","last_clean":"2024-04-24T23:14:29.295994+0000","last_became_active":"2024-04-24T23:12:28.007481+0000","last_became_peered":"2024-04-24T23:12:28.007481+0000","last_unstale":"2024-04-24T23:14:29.295994+0000","last_undegraded":"2024-04-24T23:14:29.295994+0000","last_fullsized":"2024-04-24T23:14:29.295994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:00:33.753448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.932358+0000","last_change":"2024-04-24T23:12:28.009100+0000","last_active":"2024-04-24T23:14:28.932358+0000","last_peered":"2024-04-24T23:14:28.932358+0000","last_clean":"2024-04-24T23:14:28.932358+0000","last_became_active":"2024-04-24T23:12:28.008782+0000","last_became_peered":"2024-04-24T23:12:28.008782+0000","last_unstale":"2024-04-24T23:14:28.932358+0000","last_undegraded":"2024-04-24T23:14:28.932358+0000","last_fullsized":"2024-04-24T23:14:28.932358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:44.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515607+0000","last_change":"2024-04-24T23:12:28.008129+0000","last_active":"2024-04-24T23:14:29.515607+0000","last_peered":"2024-04-24T23:14:29.515607+0000","last_clean":"2024-04-24T23:14:29.515607+0000","last_became_active":"2024-04-24T23:12:28.007819+0000","last_became_peered":"2024-04-24T23:12:28.007819+0000","last_unstale":"2024-04-24T23:14:29.515607+0000","last_undegraded":"2024-04-24T23:14:29.515607+0000","last_fullsized":"2024-04-24T23:14:29.515607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:03:25.234705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.930578+0000","last_change":"2024-04-24T23:12:27.993188+0000","last_active":"2024-04-24T23:14:28.930578+0000","last_peered":"2024-04-24T23:14:28.930578+0000","last_clean":"2024-04-24T23:14:28.930578+0000","last_became_active":"2024-04-24T23:12:27.992832+0000","last_became_peered":"2024-04-24T23:12:27.992832+0000","last_unstale":"2024-04-24T23:14:28.930578+0000","last_undegraded":"2024-04-24T23:14:28.930578+0000","last_fullsized":"2024-04-24T23:14:28.930578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:58.623164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928983+0000","last_change":"2024-04-24T23:12:28.004099+0000","last_active":"2024-04-24T23:14:28.928983+0000","last_peered":"2024-04-24T23:14:28.928983+0000","last_clean":"2024-04-24T23:14:28.928983+0000","last_became_active":"2024-04-24T23:12:28.003802+0000","last_became_peered":"2024-04-24T23:12:28.003802+0000","last_unstale":"2024-04-24T23:14:28.928983+0000","last_undegraded":"2024-04-24T23:14:28.928983+0000","last_fullsized":"2024-04-24T23:14:28.928983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:25.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.012613+0000","last_change":"2024-04-24T23:12:28.007700+0000","last_active":"2024-04-24T23:14:29.012613+0000","last_peered":"2024-04-24T23:14:29.012613+0000","last_clean":"2024-04-24T23:14:29.012613+0000","last_became_active":"2024-04-24T23:12:28.002923+0000","last_became_peered":"2024-04-24T23:12:28.002923+0000","last_unstale":"2024-04-24T23:14:29.012613+0000","last_undegraded":"2024-04-24T23:14:29.012613+0000","last_fullsized":"2024-04-24T23:14:29.012613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:46.736096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.928635+0000","last_change":"2024-04-24T23:12:28.002194+0000","last_active":"2024-04-24T23:14:28.928635+0000","last_peered":"2024-04-24T23:14:28.928635+0000","last_clean":"2024-04-24T23:14:28.928635+0000","last_became_active":"2024-04-24T23:12:28.001703+0000","last_became_peered":"2024-04-24T23:12:28.001703+0000","last_unstale":"2024-04-24T23:14:28.928635+0000","last_undegraded":"2024-04-24T23:14:28.928635+0000","last_fullsized":"2024-04-24T23:14:28.928635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:13:20.887025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515261+0000","last_change":"2024-04-24T23:12:28.015980+0000","last_active":"2024-04-24T23:14:29.515261+0000","last_peered":"2024-04-24T23:14:29.515261+0000","last_clean":"2024-04-24T23:14:29.515261+0000","last_became_active":"2024-04-24T23:12:28.013510+0000","last_became_peered":"2024-04-24T23:12:28.013510+0000","last_unstale":"2024-04-24T23:14:29.515261+0000","last_undegraded":"2024-04-24T23:14:29.515261+0000","last_fullsized":"2024-04-24T23:14:29.515261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:37.996983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:28.931126+0000","last_change":"2024-04-24T23:12:28.001883+0000","last_active":"2024-04-24T23:14:28.931126+0000","last_peered":"2024-04-24T23:14:28.931126+0000","last_clean":"2024-04-24T23:14:28.931126+0000","last_became_active":"2024-04-24T23:12:28.001499+0000","last_became_peered":"2024-04-24T23:12:28.001499+0000","last_unstale":"2024-04-24T23:14:28.931126+0000","last_undegraded":"2024-04-24T23:14:28.931126+0000","last_fullsized":"2024-04-24T23:14:28.931126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:46.805548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-24T23:14:29.515457+0000","last_change":"2024-04-24T23:12:28.004660+0000","last_active":"2024-04-24T23:14:29.515457+0000","last_peered":"2024-04-24T23:14:29.515457+0000","last_clean":"2024-04-24T23:14:29.515457+0000","last_became_active":"2024-04-24T23:12:28.004328+0000","last_became_peered":"2024-04-24T23:12:28.004328+0000","last_unstale":"2024-04-24T23:14:29.515457+0000","last_undegraded":"2024-04-24T23:14:29.515457+0000","last_fullsized":"2024-04-24T23:14:29.515457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:12:26.974476+0000","last_clean_scrub_stamp":"2024-04-24T23:12:26.974476+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:04.222202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71175,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1192,"num_read_kb":671,"num_write":254,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":180,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":217839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":525158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32908,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712244,"statfs":{"total":95995035648,"available":95961337856,"internally_reserved":0,"allocated":1773568,"data_stored":1175707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":529004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":525174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28680,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716472,"statfs":{"total":95995035648,"available":95965667328,"internally_reserved":0,"allocated":1695744,"data_stored":1122232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":520412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1150976,"data_stored":589669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":521868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":586347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":520120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":524486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1117814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:14:53.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:53.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:53 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2089425401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:53.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:53 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2617604532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.553+0000 7f7d2c49a640 1 -- 172.21.15.112:0/555741250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 msgr2=0x7f7d24100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.553+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/555741250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 0x7f7d24100aa0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009900 tx=0x7f7d2002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 -- 172.21.15.112:0/555741250 shutdown_connections 2024-04-24T23:14:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/555741250 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d24102230 0x7f7d24108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/555741250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24101870 0x7f7d24101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/555741250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 0x7f7d24100aa0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 -- 172.21.15.112:0/555741250 >> 172.21.15.112:0/555741250 conn(0x7f7d240fbbf0 msgr2=0x7f7d240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 -- 172.21.15.112:0/555741250 shutdown_connections 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 -- 172.21.15.112:0/555741250 wait complete. 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 Processor -- start 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 -- start start 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 0x7f7d24197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24101870 0x7f7d24198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d24102230 0x7f7d2419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410c430 con 0x7f7d24100680 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410c2b0 con 0x7f7d24102230 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2c49a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2410c5b0 con 0x7f7d24101870 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2a236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 0x7f7d24197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2a236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 0x7f7d24197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39728/0 (socket says 172.21.15.112:39728) 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2a236640 1 -- 172.21.15.112:0/1042160718 learned_addr learned my addr 172.21.15.112:0/1042160718 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.554+0000 7f7d2aa37640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d24102230 0x7f7d2419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2a236640 1 -- 172.21.15.112:0/1042160718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24101870 msgr2=0x7f7d24198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2a236640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24101870 0x7f7d24198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2a236640 1 -- 172.21.15.112:0/1042160718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d24102230 msgr2=0x7f7d2419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2a236640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d24102230 0x7f7d2419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2a236640 1 -- 172.21.15.112:0/1042160718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2419f740 con 0x7f7d24100680 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2aa37640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d24102230 0x7f7d2419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2a236640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 0x7f7d24197b20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009230 tx=0x7f7d20004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d1b7fe640 1 -- 172.21.15.112:0/1042160718 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2003f030 con 0x7f7d24100680 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d1b7fe640 1 -- 172.21.15.112:0/1042160718 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7d20003d00 con 0x7f7d24100680 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2419f9d0 con 0x7f7d24100680 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d1b7fe640 1 -- 172.21.15.112:0/1042160718 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20035070 con 0x7f7d24100680 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.555+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2419fec0 con 0x7f7d24100680 2024-04-24T23:14:53.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.556+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d241a00f0 con 0x7f7d24100680 2024-04-24T23:14:53.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.557+0000 7f7d1b7fe640 1 -- 172.21.15.112:0/1042160718 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7d20042030 con 0x7f7d24100680 2024-04-24T23:14:53.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.558+0000 7f7d1b7fe640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d100a2b90 0x7f7d100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.558+0000 7f7d1b7fe640 1 -- 172.21.15.112:0/1042160718 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f7d200efbe0 con 0x7f7d24100680 2024-04-24T23:14:53.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.558+0000 7f7d29a35640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d100a2b90 0x7f7d100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.558+0000 7f7d29a35640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d100a2b90 0x7f7d100a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d24199030 tx=0x7f7d14005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.562+0000 7f7d1b7fe640 1 -- 172.21.15.112:0/1042160718 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d200bc750 con 0x7f7d24100680 2024-04-24T23:14:53.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.736+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d241a08a0 con 0x7f7d24100680 2024-04-24T23:14:53.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.737+0000 7f7d1b7fe640 1 -- 172.21.15.112:0/1042160718 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7f7d200bc050 con 0x7f7d24100680 2024-04-24T23:14:53.738 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:53.738 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:53.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d100a2b90 msgr2=0x7f7d100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d100a2b90 0x7f7d100a5050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d24199030 tx=0x7f7d14005c70 comp rx=0 tx=0).stop 2024-04-24T23:14:53.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 msgr2=0x7f7d24197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 0x7f7d24197b20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009230 tx=0x7f7d20004430 comp rx=0 tx=0).stop 2024-04-24T23:14:53.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 shutdown_connections 2024-04-24T23:14:53.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f7d100a2b90 0x7f7d100a5050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d24102230 0x7f7d2419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24101870 0x7f7d24198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 --2- 172.21.15.112:0/1042160718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d24100680 0x7f7d24197b20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 >> 172.21.15.112:0/1042160718 conn(0x7f7d240fbbf0 msgr2=0x7f7d240688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 shutdown_connections 2024-04-24T23:14:53.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.739+0000 7f7d2c49a640 1 -- 172.21.15.112:0/1042160718 wait complete. 2024-04-24T23:14:53.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:14:53.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:14:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.851+0000 7f813e23f640 1 -- 172.21.15.112:0/1294804692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 msgr2=0x7f8138108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.851+0000 7f813e23f640 1 --2- 172.21.15.112:0/1294804692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 0x7f8138108ac0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8130009ca0 tx=0x7f813002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 -- 172.21.15.112:0/1294804692 shutdown_connections 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 --2- 172.21.15.112:0/1294804692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 0x7f8138108ac0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 --2- 172.21.15.112:0/1294804692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8138101870 0x7f8138101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 --2- 172.21.15.112:0/1294804692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8138100680 0x7f8138100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 -- 172.21.15.112:0/1294804692 >> 172.21.15.112:0/1294804692 conn(0x7f81380fbbf0 msgr2=0x7f81380fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 -- 172.21.15.112:0/1294804692 shutdown_connections 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 -- 172.21.15.112:0/1294804692 wait complete. 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 Processor -- start 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 -- start start 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8138100680 0x7f8138197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8138101870 0x7f8138198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 0x7f813819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f813810c430 con 0x7f8138102230 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f813810c2b0 con 0x7f8138100680 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f813e23f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f813810c5b0 con 0x7f8138101870 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.852+0000 7f8137fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 0x7f813819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f8137fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 0x7f813819f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39752/0 (socket says 172.21.15.112:39752) 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f8137fff640 1 -- 172.21.15.112:0/4176144512 learned_addr learned my addr 172.21.15.112:0/4176144512 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f812effd640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8138101870 0x7f8138198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f81377fe640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8138100680 0x7f8138197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f8137fff640 1 -- 172.21.15.112:0/4176144512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8138101870 msgr2=0x7f8138198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f8137fff640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8138101870 0x7f8138198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f8137fff640 1 -- 172.21.15.112:0/4176144512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8138100680 msgr2=0x7f8138197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f8137fff640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8138100680 0x7f8138197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f8137fff640 1 -- 172.21.15.112:0/4176144512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813819f760 con 0x7f8138102230 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f8137fff640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 0x7f813819f0f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8130031a40 tx=0x7f8130031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f81357fa640 1 -- 172.21.15.112:0/4176144512 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813002fbe0 con 0x7f8138102230 2024-04-24T23:14:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f81357fa640 1 -- 172.21.15.112:0/4176144512 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f813002fd80 con 0x7f8138102230 2024-04-24T23:14:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813819f9f0 con 0x7f8138102230 2024-04-24T23:14:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f81357fa640 1 -- 172.21.15.112:0/4176144512 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130046980 con 0x7f8138102230 2024-04-24T23:14:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.853+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813819ff40 con 0x7f8138102230 2024-04-24T23:14:53.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.854+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81381a0170 con 0x7f8138102230 2024-04-24T23:14:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.855+0000 7f81357fa640 1 -- 172.21.15.112:0/4176144512 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8130030ce0 con 0x7f8138102230 2024-04-24T23:14:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.856+0000 7f81357fa640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f81240a2b90 0x7f81240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.856+0000 7f81377fe640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f81240a2b90 0x7f81240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.856+0000 7f81357fa640 1 -- 172.21.15.112:0/4176144512 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f81300f0a60 con 0x7f8138102230 2024-04-24T23:14:53.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.856+0000 7f81377fe640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f81240a2b90 0x7f81240a5050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8128005fd0 tx=0x7f8128005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:53.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:53.860+0000 7f81357fa640 1 -- 172.21.15.112:0/4176144512 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f81300bd660 con 0x7f8138102230 2024-04-24T23:14:54.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.035+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81381a0460 con 0x7f8138102230 2024-04-24T23:14:54.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.036+0000 7f81357fa640 1 -- 172.21.15.112:0/4176144512 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f81300bcf60 con 0x7f8138102230 2024-04-24T23:14:54.037 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f81240a2b90 msgr2=0x7f81240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f81240a2b90 0x7f81240a5050 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8128005fd0 tx=0x7f8128005850 comp rx=0 tx=0).stop 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 msgr2=0x7f813819f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 0x7f813819f0f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8130031a40 tx=0x7f8130031ee0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 shutdown_connections 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f81240a2b90 0x7f81240a5050 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8138102230 0x7f813819f0f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8138101870 0x7f8138198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 --2- 172.21.15.112:0/4176144512 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8138100680 0x7f8138197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 >> 172.21.15.112:0/4176144512 conn(0x7f81380fbbf0 msgr2=0x7f81380fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:54.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 shutdown_connections 2024-04-24T23:14:54.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.038+0000 7f813e23f640 1 -- 172.21.15.112:0/4176144512 wait complete. 2024-04-24T23:14:54.050 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:14:54.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 -- 172.21.15.112:0/889495074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8102240 msgr2=0x7fbee8108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/889495074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8102240 0x7fbee8108ad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009ca0 tx=0x7fbee002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 -- 172.21.15.112:0/889495074 shutdown_connections 2024-04-24T23:14:54.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/889495074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8102240 0x7fbee8108ad0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/889495074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee8101880 0x7fbee8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/889495074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee8100690 0x7fbee8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 -- 172.21.15.112:0/889495074 >> 172.21.15.112:0/889495074 conn(0x7fbee80fbc00 msgr2=0x7fbee80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 -- 172.21.15.112:0/889495074 shutdown_connections 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 -- 172.21.15.112:0/889495074 wait complete. 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.150+0000 7fbeeeb68640 1 Processor -- start 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeeeb68640 1 -- start start 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeeeb68640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee8100690 0x7fbee8104620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeeeb68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8101880 0x7fbee8102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeeeb68640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee8102240 0x7fbee81031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeeeb68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbee810c440 con 0x7fbee8101880 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeeeb68640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbee810c2c0 con 0x7fbee8102240 2024-04-24T23:14:54.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeeeb68640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbee810c5c0 con 0x7fbee8100690 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8101880 0x7fbee8102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8101880 0x7fbee8102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39774/0 (socket says 172.21.15.112:39774) 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 -- 172.21.15.112:0/1008145391 learned_addr learned my addr 172.21.15.112:0/1008145391 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeec904640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee8100690 0x7fbee8104620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeed105640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee8102240 0x7fbee81031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 -- 172.21.15.112:0/1008145391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee8100690 msgr2=0x7fbee8104620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee8100690 0x7fbee8104620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 -- 172.21.15.112:0/1008145391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee8102240 msgr2=0x7fbee81031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee8102240 0x7fbee81031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 -- 172.21.15.112:0/1008145391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee8103a30 con 0x7fbee8101880 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeec904640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee8100690 0x7fbee8104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbedffff640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8101880 0x7fbee8102c60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbed800d770 tx=0x7fbed800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeddffb640 1 -- 172.21.15.112:0/1008145391 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8017070 con 0x7fbee8101880 2024-04-24T23:14:54.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee81a1c60 con 0x7fbee8101880 2024-04-24T23:14:54.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeddffb640 1 -- 172.21.15.112:0/1008145391 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbed8005520 con 0x7fbee8101880 2024-04-24T23:14:54.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.151+0000 7fbeddffb640 1 -- 172.21.15.112:0/1008145391 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8010460 con 0x7fbee8101880 2024-04-24T23:14:54.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.152+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee81a21b0 con 0x7fbee8101880 2024-04-24T23:14:54.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.153+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee81a2380 con 0x7fbee8101880 2024-04-24T23:14:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.153+0000 7fbeddffb640 1 -- 172.21.15.112:0/1008145391 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbed8004ea0 con 0x7fbee8101880 2024-04-24T23:14:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.154+0000 7fbeddffb640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed00a2b90 0x7fbed00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.154+0000 7fbeec904640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed00a2b90 0x7fbed00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.154+0000 7fbeddffb640 1 -- 172.21.15.112:0/1008145391 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbed80cf030 con 0x7fbee8101880 2024-04-24T23:14:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.155+0000 7fbeec904640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed00a2b90 0x7fbed00a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbee40058c0 tx=0x7fbee4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.159+0000 7fbeddffb640 1 -- 172.21.15.112:0/1008145391 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbed809bba0 con 0x7fbee8101880 2024-04-24T23:14:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:54 smithi150 ceph-mon[32247]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:54 smithi150 ceph-mon[32247]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-24T23:14:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:54 smithi150 ceph-mon[32247]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:54 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1042160718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:54 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4176144512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:54.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.332+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbee81a2550 con 0x7fbee8101880 2024-04-24T23:14:54.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.333+0000 7fbeddffb640 1 -- 172.21.15.112:0/1008145391 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fbed809b4a0 con 0x7fbee8101880 2024-04-24T23:14:54.334 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:54.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed00a2b90 msgr2=0x7fbed00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed00a2b90 0x7fbed00a5050 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbee40058c0 tx=0x7fbee4005850 comp rx=0 tx=0).stop 2024-04-24T23:14:54.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8101880 msgr2=0x7fbee8102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8101880 0x7fbee8102c60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbed800d770 tx=0x7fbed800dc30 comp rx=0 tx=0).stop 2024-04-24T23:14:54.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 shutdown_connections 2024-04-24T23:14:54.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbed00a2b90 0x7fbed00a5050 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbee8102240 0x7fbee81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee8101880 0x7fbee8102c60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 --2- 172.21.15.112:0/1008145391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbee8100690 0x7fbee8104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 >> 172.21.15.112:0/1008145391 conn(0x7fbee80fbc00 msgr2=0x7fbee80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 shutdown_connections 2024-04-24T23:14:54.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.335+0000 7fbeeeb68640 1 -- 172.21.15.112:0/1008145391 wait complete. 2024-04-24T23:14:54.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:54 smithi112 ceph-mon[29186]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:54.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:54 smithi112 ceph-mon[29186]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-24T23:14:54.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:54 smithi112 ceph-mon[29186]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:54.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:54 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1042160718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:54.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:54 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4176144512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:54.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:14:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 -- 172.21.15.112:0/3919922373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 msgr2=0x7fc630100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 --2- 172.21.15.112:0/3919922373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 0x7fc630100aa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009ca0 tx=0x7fc62c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 -- 172.21.15.112:0/3919922373 shutdown_connections 2024-04-24T23:14:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 --2- 172.21.15.112:0/3919922373 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc630102230 0x7fc630108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 --2- 172.21.15.112:0/3919922373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc630101870 0x7fc630101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 --2- 172.21.15.112:0/3919922373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 0x7fc630100aa0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 -- 172.21.15.112:0/3919922373 >> 172.21.15.112:0/3919922373 conn(0x7fc6300fbbf0 msgr2=0x7fc6300fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 -- 172.21.15.112:0/3919922373 shutdown_connections 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.447+0000 7fc638568640 1 -- 172.21.15.112:0/3919922373 wait complete. 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc638568640 1 Processor -- start 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc638568640 1 -- start start 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc638568640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 0x7fc630197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc638568640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc630101870 0x7fc630198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc638568640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc630102230 0x7fc63019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc638568640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc63010c430 con 0x7fc630100680 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc638568640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc63010c2b0 con 0x7fc630101870 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc638568640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc63010c5b0 con 0x7fc630102230 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636304640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 0x7fc630197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636304640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 0x7fc630197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39800/0 (socket says 172.21.15.112:39800) 2024-04-24T23:14:54.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636304640 1 -- 172.21.15.112:0/3356345896 learned_addr learned my addr 172.21.15.112:0/3356345896 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636b05640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc630102230 0x7fc63019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc635b03640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc630101870 0x7fc630198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636304640 1 -- 172.21.15.112:0/3356345896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc630102230 msgr2=0x7fc63019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636304640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc630102230 0x7fc63019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636304640 1 -- 172.21.15.112:0/3356345896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc630101870 msgr2=0x7fc630198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636304640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc630101870 0x7fc630198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636304640 1 -- 172.21.15.112:0/3356345896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63019f800 con 0x7fc630100680 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc635b03640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc630101870 0x7fc630198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.448+0000 7fc636b05640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc630102230 0x7fc63019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.449+0000 7fc636304640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 0x7fc630197be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc62c000f80 tx=0x7fc62c02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.449+0000 7fc6277fe640 1 -- 172.21.15.112:0/3356345896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c02fdc0 con 0x7fc630100680 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.449+0000 7fc6277fe640 1 -- 172.21.15.112:0/3356345896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc62c031df0 con 0x7fc630100680 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.449+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63019fa90 con 0x7fc630100680 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.449+0000 7fc6277fe640 1 -- 172.21.15.112:0/3356345896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c046980 con 0x7fc630100680 2024-04-24T23:14:54.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.449+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63019ffe0 con 0x7fc630100680 2024-04-24T23:14:54.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.450+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6301a0210 con 0x7fc630100680 2024-04-24T23:14:54.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.451+0000 7fc6277fe640 1 -- 172.21.15.112:0/3356345896 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc62c0305c0 con 0x7fc630100680 2024-04-24T23:14:54.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.452+0000 7fc6277fe640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc61c0a2b90 0x7fc61c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.452+0000 7fc635b03640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc61c0a2b90 0x7fc61c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.452+0000 7fc6277fe640 1 -- 172.21.15.112:0/3356345896 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc62c0f0b10 con 0x7fc630100680 2024-04-24T23:14:54.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.452+0000 7fc635b03640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc61c0a2b90 0x7fc61c0a5050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc6301990f0 tx=0x7fc620005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:54.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.456+0000 7fc6277fe640 1 -- 172.21.15.112:0/3356345896 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc62c0bd680 con 0x7fc630100680 2024-04-24T23:14:54.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:54 smithi187 ceph-mon[33292]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:54.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:54 smithi187 ceph-mon[33292]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-24T23:14:54.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:54 smithi187 ceph-mon[33292]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:54.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:54 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1042160718' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:54.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:54 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4176144512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:14:54.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.634+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc6301a03e0 con 0x7fc630100680 2024-04-24T23:14:54.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.634+0000 7fc6277fe640 1 -- 172.21.15.112:0/3356345896 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fc62c0bcf80 con 0x7fc630100680 2024-04-24T23:14:54.635 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:14:54.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc61c0a2b90 msgr2=0x7fc61c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc61c0a2b90 0x7fc61c0a5050 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc6301990f0 tx=0x7fc620005e50 comp rx=0 tx=0).stop 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 msgr2=0x7fc630197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 0x7fc630197be0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc62c000f80 tx=0x7fc62c02f840 comp rx=0 tx=0).stop 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 shutdown_connections 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc61c0a2b90 0x7fc61c0a5050 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc630102230 0x7fc63019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc630101870 0x7fc630198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 --2- 172.21.15.112:0/3356345896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc630100680 0x7fc630197be0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.636+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 >> 172.21.15.112:0/3356345896 conn(0x7fc6300fbbf0 msgr2=0x7fc6300fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.637+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 shutdown_connections 2024-04-24T23:14:54.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.637+0000 7fc638568640 1 -- 172.21.15.112:0/3356345896 wait complete. 2024-04-24T23:14:54.649 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- 172.21.15.112:0/835207153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124101870 msgr2=0x7fc124101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 --2- 172.21.15.112:0/835207153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124101870 0x7fc124101cf0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc110009900 tx=0x7fc11002a410 comp rx=0 tx=0).stop 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- 172.21.15.112:0/835207153 shutdown_connections 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 --2- 172.21.15.112:0/835207153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102230 0x7fc124108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 --2- 172.21.15.112:0/835207153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124101870 0x7fc124101cf0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 --2- 172.21.15.112:0/835207153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124100680 0x7fc124100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- 172.21.15.112:0/835207153 >> 172.21.15.112:0/835207153 conn(0x7fc1240fbc30 msgr2=0x7fc1240fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- 172.21.15.112:0/835207153 shutdown_connections 2024-04-24T23:14:54.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- 172.21.15.112:0/835207153 wait complete. 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 Processor -- start 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- start start 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124100680 0x7fc124196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124101870 0x7fc124197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102230 0x7fc12419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410c4d0 con 0x7fc124100680 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410c350 con 0x7fc124101870 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc12b14f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc12410c650 con 0x7fc124102230 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc128eeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124100680 0x7fc124196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc128eeb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124100680 0x7fc124196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39828/0 (socket says 172.21.15.112:39828) 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.748+0000 7fc128eeb640 1 -- 172.21.15.112:0/2928495332 learned_addr learned my addr 172.21.15.112:0/2928495332 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc1296ec640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102230 0x7fc12419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc123fff640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124101870 0x7fc124197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc128eeb640 1 -- 172.21.15.112:0/2928495332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102230 msgr2=0x7fc12419e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc128eeb640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102230 0x7fc12419e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc128eeb640 1 -- 172.21.15.112:0/2928495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124101870 msgr2=0x7fc124197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc128eeb640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124101870 0x7fc124197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc128eeb640 1 -- 172.21.15.112:0/2928495332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12419e730 con 0x7fc124100680 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc128eeb640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124100680 0x7fc124196b10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc11800d920 tx=0x7fc11800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc121ffb640 1 -- 172.21.15.112:0/2928495332 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11801b070 con 0x7fc124100680 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc121ffb640 1 -- 172.21.15.112:0/2928495332 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc11800edc0 con 0x7fc124100680 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc124069de0 con 0x7fc124100680 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc121ffb640 1 -- 172.21.15.112:0/2928495332 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180148b0 con 0x7fc124100680 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.749+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12406a380 con 0x7fc124100680 2024-04-24T23:14:54.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.750+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc124107410 con 0x7fc124100680 2024-04-24T23:14:54.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.751+0000 7fc121ffb640 1 -- 172.21.15.112:0/2928495332 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc11801e070 con 0x7fc124100680 2024-04-24T23:14:54.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.752+0000 7fc121ffb640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2c10 0x7fc1140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:54.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.752+0000 7fc123fff640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2c10 0x7fc1140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:54.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.752+0000 7fc121ffb640 1 -- 172.21.15.112:0/2928495332 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc1180d0b50 con 0x7fc124100680 2024-04-24T23:14:54.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.752+0000 7fc123fff640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2c10 0x7fc1140a50d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc110006010 tx=0x7fc110031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:54.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.755+0000 7fc121ffb640 1 -- 172.21.15.112:0/2928495332 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc11809d010 con 0x7fc124100680 2024-04-24T23:14:54.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.933+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc12419e900 con 0x7fc124100680 2024-04-24T23:14:54.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.933+0000 7fc121ffb640 1 -- 172.21.15.112:0/2928495332 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fc1180a1310 con 0x7fc124100680 2024-04-24T23:14:54.935 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2c10 msgr2=0x7fc1140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2c10 0x7fc1140a50d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc110006010 tx=0x7fc110031640 comp rx=0 tx=0).stop 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124100680 msgr2=0x7fc124196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124100680 0x7fc124196b10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc11800d920 tx=0x7fc11800dde0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 shutdown_connections 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2c10 0x7fc1140a50d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc124102230 0x7fc12419e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc124101870 0x7fc124197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.937 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 --2- 172.21.15.112:0/2928495332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc124100680 0x7fc124196b10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:54.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 >> 172.21.15.112:0/2928495332 conn(0x7fc1240fbc30 msgr2=0x7fc124104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:54.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 shutdown_connections 2024-04-24T23:14:54.938 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:54.935+0000 7fc12b14f640 1 -- 172.21.15.112:0/2928495332 wait complete. 2024-04-24T23:14:54.948 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.044+0000 7f780ee88640 1 -- 172.21.15.112:0/1640788792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808101870 msgr2=0x7f7808101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.044+0000 7f780ee88640 1 --2- 172.21.15.112:0/1640788792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808101870 0x7f7808101cf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009900 tx=0x7f77f402a410 comp rx=0 tx=0).stop 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 -- 172.21.15.112:0/1640788792 shutdown_connections 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 --2- 172.21.15.112:0/1640788792 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7808102230 0x7f7808108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 --2- 172.21.15.112:0/1640788792 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808101870 0x7f7808101cf0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 --2- 172.21.15.112:0/1640788792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7808100680 0x7f7808100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 -- 172.21.15.112:0/1640788792 >> 172.21.15.112:0/1640788792 conn(0x7f78080fbbf0 msgr2=0x7f78080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 -- 172.21.15.112:0/1640788792 shutdown_connections 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 -- 172.21.15.112:0/1640788792 wait complete. 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 Processor -- start 2024-04-24T23:14:55.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 -- start start 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7808100680 0x7f7808197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7808101870 0x7f78081980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808102230 0x7f780819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f780810c430 con 0x7f7808102230 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f780810c2b0 con 0x7f7808101870 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780ee88640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f780810c5b0 con 0x7f7808100680 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780d425640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808102230 0x7f780819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780d425640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808102230 0x7f780819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39838/0 (socket says 172.21.15.112:39838) 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.045+0000 7f780d425640 1 -- 172.21.15.112:0/595080901 learned_addr learned my addr 172.21.15.112:0/595080901 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f7807fff640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7808101870 0x7f78081980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780cc24640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7808100680 0x7f7808197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780d425640 1 -- 172.21.15.112:0/595080901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7808100680 msgr2=0x7f7808197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780d425640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7808100680 0x7f7808197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780d425640 1 -- 172.21.15.112:0/595080901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7808101870 msgr2=0x7f78081980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780d425640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7808101870 0x7f78081980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780d425640 1 -- 172.21.15.112:0/595080901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780819f7a0 con 0x7f7808102230 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780d425640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808102230 0x7f780819f130 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7800009fd0 tx=0x7f780000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f7805ffb640 1 -- 172.21.15.112:0/595080901 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800017070 con 0x7f7808102230 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f7805ffb640 1 -- 172.21.15.112:0/595080901 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f7800004020 con 0x7f7808102230 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780819fa90 con 0x7f7808102230 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f7805ffb640 1 -- 172.21.15.112:0/595080901 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000106c0 con 0x7f7808102230 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.046+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780819ff60 con 0x7f7808102230 2024-04-24T23:14:55.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.047+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78081a0190 con 0x7f7808102230 2024-04-24T23:14:55.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.048+0000 7f7805ffb640 1 -- 172.21.15.112:0/595080901 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f78000041d0 con 0x7f7808102230 2024-04-24T23:14:55.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.049+0000 7f7805ffb640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77f80a2b90 0x7f77f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.049+0000 7f7805ffb640 1 -- 172.21.15.112:0/595080901 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f78000cf1f0 con 0x7f7808102230 2024-04-24T23:14:55.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.049+0000 7f780cc24640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77f80a2b90 0x7f77f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.049+0000 7f780cc24640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77f80a2b90 0x7f77f80a5050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f78081016d0 tx=0x7f77fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.055+0000 7f7805ffb640 1 -- 172.21.15.112:0/595080901 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f780009bd60 con 0x7f7808102230 2024-04-24T23:14:55.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.267+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78081a0520 con 0x7f7808102230 2024-04-24T23:14:55.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.268+0000 7f7805ffb640 1 -- 172.21.15.112:0/595080901 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f780009b660 con 0x7f7808102230 2024-04-24T23:14:55.270 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:55.270 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:14:48.128683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24424,24425,24490,24524,24482]},"id":1}]} 2024-04-24T23:14:55.270 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77f80a2b90 msgr2=0x7f77f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77f80a2b90 0x7f77f80a5050 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f78081016d0 tx=0x7f77fc005f90 comp rx=0 tx=0).stop 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808102230 msgr2=0x7f780819f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808102230 0x7f780819f130 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7800009fd0 tx=0x7f780000b260 comp rx=0 tx=0).stop 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 shutdown_connections 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f77f80a2b90 0x7f77f80a5050 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7808102230 0x7f780819f130 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7808101870 0x7f78081980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 --2- 172.21.15.112:0/595080901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7808100680 0x7f7808197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 >> 172.21.15.112:0/595080901 conn(0x7f78080fbbf0 msgr2=0x7f78080688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 shutdown_connections 2024-04-24T23:14:55.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.270+0000 7f780ee88640 1 -- 172.21.15.112:0/595080901 wait complete. 2024-04-24T23:14:55.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:55 smithi112 ceph-mon[29186]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:55.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:55 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1008145391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:55 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3356345896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:55 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2928495332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:14:55.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:55 smithi150 ceph-mon[32247]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:55.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:55 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1008145391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:55 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3356345896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:55 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2928495332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 -- 172.21.15.112:0/736590684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8101890 msgr2=0x7fc4a8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/736590684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8101890 0x7fc4a8101d10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc498009900 tx=0x7fc49802a410 comp rx=0 tx=0).stop 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 -- 172.21.15.112:0/736590684 shutdown_connections 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/736590684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a8102250 0x7fc4a8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/736590684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a8101890 0x7fc4a8101d10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/736590684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a81006a0 0x7fc4a8100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 -- 172.21.15.112:0/736590684 >> 172.21.15.112:0/736590684 conn(0x7fc4a80fbc30 msgr2=0x7fc4a80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 -- 172.21.15.112:0/736590684 shutdown_connections 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.379+0000 7fc4af8a5640 1 -- 172.21.15.112:0/736590684 wait complete. 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4af8a5640 1 Processor -- start 2024-04-24T23:14:55.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4af8a5640 1 -- start start 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4af8a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a81006a0 0x7fc4a8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4af8a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a8101890 0x7fc4a81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4af8a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a8102250 0x7fc4a819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4af8a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810c450 con 0x7fc4a81006a0 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4af8a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810c2d0 con 0x7fc4a8102250 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4af8a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a810c5d0 con 0x7fc4a8101890 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a81006a0 0x7fc4a8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a81006a0 0x7fc4a8197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39866/0 (socket says 172.21.15.112:39866) 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 -- 172.21.15.112:0/2907853041 learned_addr learned my addr 172.21.15.112:0/2907853041 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ade42640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a8102250 0x7fc4a819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ace40640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a8101890 0x7fc4a81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 -- 172.21.15.112:0/2907853041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a8101890 msgr2=0x7fc4a81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a8101890 0x7fc4a81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 -- 172.21.15.112:0/2907853041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a8102250 msgr2=0x7fc4a819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a8102250 0x7fc4a819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 -- 172.21.15.112:0/2907853041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a819f7c0 con 0x7fc4a81006a0 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.380+0000 7fc4ad641640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a81006a0 0x7fc4a8197ba0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400d950 tx=0x7fc4a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.381+0000 7fc49e7fc640 1 -- 172.21.15.112:0/2907853041 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4017020 con 0x7fc4a81006a0 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.381+0000 7fc49e7fc640 1 -- 172.21.15.112:0/2907853041 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc4a4009c30 con 0x7fc4a81006a0 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.381+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a819fab0 con 0x7fc4a81006a0 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.381+0000 7fc49e7fc640 1 -- 172.21.15.112:0/2907853041 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a400f7e0 con 0x7fc4a81006a0 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.381+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a819ffd0 con 0x7fc4a81006a0 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.382+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a80689b0 con 0x7fc4a81006a0 2024-04-24T23:14:55.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.383+0000 7fc49e7fc640 1 -- 172.21.15.112:0/2907853041 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc4a401a020 con 0x7fc4a81006a0 2024-04-24T23:14:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.383+0000 7fc49e7fc640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc4940a2b90 0x7fc4940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.383+0000 7fc4ace40640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc4940a2b90 0x7fc4940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.383+0000 7fc49e7fc640 1 -- 172.21.15.112:0/2907853041 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc4a40ceed0 con 0x7fc4a81006a0 2024-04-24T23:14:55.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.384+0000 7fc4ace40640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc4940a2b90 0x7fc4940a5050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc498000c00 tx=0x7fc498031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.388+0000 7fc49e7fc640 1 -- 172.21.15.112:0/2907853041 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc4a4021070 con 0x7fc4a81006a0 2024-04-24T23:14:55.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:55 smithi187 ceph-mon[33292]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:14:55.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:55 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1008145391' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:55 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3356345896' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:55 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2928495332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:14:55.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.565+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4a81a0260 con 0x7fc4a81006a0 2024-04-24T23:14:55.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.567+0000 7fc49e7fc640 1 -- 172.21.15.112:0/2907853041 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7fc4a4081190 con 0x7fc4a81006a0 2024-04-24T23:14:55.569 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:14:55.569 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc4940a2b90 msgr2=0x7fc4940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc4940a2b90 0x7fc4940a5050 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc498000c00 tx=0x7fc498031640 comp rx=0 tx=0).stop 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a81006a0 msgr2=0x7fc4a8197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a81006a0 0x7fc4a8197ba0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400d950 tx=0x7fc4a400de10 comp rx=0 tx=0).stop 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 shutdown_connections 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc4940a2b90 0x7fc4940a5050 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a8102250 0x7fc4a819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a8101890 0x7fc4a81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 --2- 172.21.15.112:0/2907853041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4a81006a0 0x7fc4a8197ba0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 >> 172.21.15.112:0/2907853041 conn(0x7fc4a80fbc30 msgr2=0x7fc4a80fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 shutdown_connections 2024-04-24T23:14:55.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.569+0000 7fc4af8a5640 1 -- 172.21.15.112:0/2907853041 wait complete. 2024-04-24T23:14:55.582 DEBUG:tasks.fwd_scrub:Yielding 2024-04-24T23:14:55.583 INFO:teuthology.run_tasks:Running task workunit... 2024-04-24T23:14:55.597 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:14:55.597 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T23:14:55.598 INFO:tasks.workunit:timeout=3h 2024-04-24T23:14:55.598 INFO:tasks.workunit:cleanup=True 2024-04-24T23:14:55.598 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:14:55.598 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-24T23:14:55.599 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:14:55.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:14:55.599 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:14:55.613 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T23:14:55.613 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-24T23:14:55.613 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-24T23:14:55.613 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:14:55.613 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-24T23:14:55.614 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-24 23:12:55.247654038 +0000 2024-04-24T23:14:55.614 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-24 23:12:55.247654038 +0000 2024-04-24T23:14:55.614 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-24 23:14:47.452193903 +0000 2024-04-24T23:14:55.614 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-24 23:12:55.247654038 +0000 2024-04-24T23:14:55.614 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T23:14:55.614 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T23:14:55.685 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:14:55.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 -- 172.21.15.112:0/2310201408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 msgr2=0x7fc5e8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/2310201408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 0x7fc5e8109c70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009cd0 tx=0x7fc5e002a8e0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 -- 172.21.15.112:0/2310201408 shutdown_connections 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/2310201408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 0x7fc5e8109c70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/2310201408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5e8102a20 0x7fc5e8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/2310201408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e80ffa50 0x7fc5e80ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 -- 172.21.15.112:0/2310201408 >> 172.21.15.112:0/2310201408 conn(0x7fc5e80fb380 msgr2=0x7fc5e80fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 -- 172.21.15.112:0/2310201408 shutdown_connections 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.702+0000 7fc5efbb0640 1 -- 172.21.15.112:0/2310201408 wait complete. 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5efbb0640 1 Processor -- start 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5efbb0640 1 -- start start 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5efbb0640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5e80ffa50 0x7fc5e8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5efbb0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e8102a20 0x7fc5e8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5efbb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 0x7fc5e819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5efbb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e810d560 con 0x7fc5e81033e0 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5efbb0640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e810d3e0 con 0x7fc5e80ffa50 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5efbb0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e810d6e0 con 0x7fc5e8102a20 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ee14d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 0x7fc5e819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ee14d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 0x7fc5e819f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39882/0 (socket says 172.21.15.112:39882) 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ee14d640 1 -- 172.21.15.112:0/3588887979 learned_addr learned my addr 172.21.15.112:0/3588887979 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5e80ffa50 0x7fc5e8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e8102a20 0x7fc5e8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ee14d640 1 -- 172.21.15.112:0/3588887979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e8102a20 msgr2=0x7fc5e8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ee14d640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e8102a20 0x7fc5e8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ee14d640 1 -- 172.21.15.112:0/3588887979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5e80ffa50 msgr2=0x7fc5e8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ee14d640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5e80ffa50 0x7fc5e8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.703+0000 7fc5ee14d640 1 -- 172.21.15.112:0/3588887979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e819f710 con 0x7fc5e81033e0 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.704+0000 7fc5ee14d640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 0x7fc5e819f0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009df0 tx=0x7fc5e0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.704+0000 7fc5deffd640 1 -- 172.21.15.112:0/3588887979 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0004240 con 0x7fc5e81033e0 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.704+0000 7fc5efbb0640 1 -- 172.21.15.112:0/3588887979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e819f9a0 con 0x7fc5e81033e0 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.704+0000 7fc5deffd640 1 -- 172.21.15.112:0/3588887979 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5e00305a0 con 0x7fc5e81033e0 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.704+0000 7fc5deffd640 1 -- 172.21.15.112:0/3588887979 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00466c0 con 0x7fc5e81033e0 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.704+0000 7fc5efbb0640 1 -- 172.21.15.112:0/3588887979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e819fef0 con 0x7fc5e81033e0 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.706+0000 7fc5deffd640 1 -- 172.21.15.112:0/3588887979 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc5e0030b50 con 0x7fc5e81033e0 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.706+0000 7fc5f29a0740 1 Processor -- start 2024-04-24T23:14:55.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5f29a0740 1 -- start start 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 0x55a67bf67690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf6a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a67bec12c0 con 0x55a67bf67d00 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a67be69910 con 0x55a67bf62b90 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a67bebd280 con 0x55a67bf621a0 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5deffd640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5d40a2b90 0x7fc5d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf6a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:48946/0 (socket says 172.21.15.112:48946) 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 -- 172.21.15.112:0/1851642875 learned_addr learned my addr 172.21.15.112:0/1851642875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ee14d640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5d40a2b90 0x7fc5d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/1851642875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 0x55a67bf67690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5deffd640 1 -- 172.21.15.112:0/3588887979 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc5e002e030 con 0x7fc5e81033e0 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 -- 172.21.15.112:0/1851642875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 msgr2=0x55a67bf67690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/1851642875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 0x55a67bf67690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 -- 172.21.15.112:0/1851642875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 msgr2=0x55a67bf6a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/1851642875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf6a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.707+0000 7fc5ed14b640 1 -- 172.21.15.112:0/1851642875 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67be7afb0 con 0x55a67bf621a0 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.708+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/1851642875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf625c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8012790 tx=0x7fc5d800b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fffaefd7675a66f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.708+0000 7fc5ee14d640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5d40a2b90 0x7fc5d40a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00b9ad0 tx=0x7fc5e008f110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.708+0000 7fc5d3fff640 1 -- 172.21.15.112:0/1851642875 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8004050 con 0x55a67bf621a0 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.708+0000 7fc5d3fff640 1 -- 172.21.15.112:0/1851642875 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5d8005520 con 0x55a67bf621a0 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.708+0000 7fc5d3fff640 1 -- 172.21.15.112:0/1851642875 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8004d00 con 0x55a67bf621a0 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1851642875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 msgr2=0x55a67bf625c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1851642875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf625c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8012790 tx=0x7fc5d800b390 comp rx=0 tx=0).stop 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1851642875 shutdown_connections 2024-04-24T23:14:55.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1851642875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf6a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1851642875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 0x55a67bf67690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1851642875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf625c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1851642875 >> 172.21.15.112:0/1851642875 conn(0x55a67bf25950 msgr2=0x55a67bf59810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1851642875 shutdown_connections 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1851642875 wait complete. 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 -- start start 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 Processor -- start 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.709+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 0x55a67bf6cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf67d00 0x55a67bf73ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a67bf6acb0 con 0x55a67bf621a0 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a67be7ce90 con 0x55a67bf67d00 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a67bf6ae30 con 0x55a67bf62b90 2024-04-24T23:14:55.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5f29a0740 10 client.0 _ll_register_callbacks cb 0x55a67beb8970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39908/0 (socket says 172.21.15.112:39908) 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 -- 172.21.15.112:0/1333331157 learned_addr learned my addr 172.21.15.112:0/1333331157 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 0x55a67bf6cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 -- 172.21.15.112:0/1333331157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 msgr2=0x55a67bf6cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 0x55a67bf6cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 -- 172.21.15.112:0/1333331157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf67d00 msgr2=0x55a67bf73ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf67d00 0x55a67bf73ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 -- 172.21.15.112:0/1333331157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67bc9d7e0 con 0x55a67bf621a0 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6c620 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8005bc0 tx=0x7fc5d8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84781961bfea5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5d17fa640 1 -- 172.21.15.112:0/1333331157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d802c060 con 0x55a67bf621a0 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5d17fa640 1 -- 172.21.15.112:0/1333331157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc5d8004c80 con 0x55a67bf621a0 2024-04-24T23:14:55.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1333331157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d8005520 con 0x55a67bf621a0 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5d17fa640 1 -- 172.21.15.112:0/1333331157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5d8033050 con 0x55a67bf621a0 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5f29a0740 10 client.15669 fetch_fsmap learned FSMap version 28 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.710+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1333331157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a67bcfdf70 con 0x55a67bf621a0 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5d17fa640 1 -- 172.21.15.112:0/1333331157 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5d8005520 con 0x55a67bf621a0 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5d17fa640 1 -- 172.21.15.112:0/1333331157 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc5d8026030 con 0x55a67bf621a0 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5f29a0740 10 client.15669 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5f29a0740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5f29a0740 20 client.15669 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55a67bf77ea0 0x55a67bf7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5f29a0740 4 client.15669 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1333331157 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a67be955a0 con 0x55a67bf77ea0 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.711+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55a67bf77ea0 0x55a67bf7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.712+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55a67bf77ea0 0x55a67bf7a2c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bee06db30505615f server_cookie=458e5527f0b96382 in_seq=0 out_seq=0 2024-04-24T23:14:55.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.712+0000 7fc5d17fa640 10 client.15669 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.712+0000 7fc5d17fa640 1 -- 172.21.15.112:0/1333331157 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a67be955a0 con 0x55a67bf77ea0 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.712+0000 7fc5d17fa640 10 client.15669 handle_command_reply: tid=0 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 client.15669 shutdown 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1333331157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 msgr2=0x55a67bf6c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6c620 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8005bc0 tx=0x7fc5d8005230 comp rx=0 tx=0).stop 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1333331157 shutdown_connections 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55a67bf77ea0 0x55a67bf7a2c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf67d00 0x55a67bf73ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 0x55a67bf6cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5d17fa640 0 client.15669 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/1333331157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6c620 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1333331157 >> 172.21.15.112:0/1333331157 conn(0x55a67bf25950 msgr2=0x55a67bf58950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1333331157 shutdown_connections 2024-04-24T23:14:55.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 1 -- 172.21.15.112:0/1333331157 wait complete. 2024-04-24T23:14:55.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.713+0000 7fc5f29a0740 20 client.15669 trim_cache size 0 max 16384 2024-04-24T23:14:55.722 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T23:14:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.730+0000 7fc5f29a0740 1 Processor -- start 2024-04-24T23:14:55.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5f29a0740 1 -- start start 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 0x55a67bf73b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf67d00 0x55a67bf5ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81a8330 con 0x55a67bf621a0 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81a9e70 con 0x55a67bf67d00 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81a8a00 con 0x55a67bf62b90 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39920/0 (socket says 172.21.15.112:39920) 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 -- 172.21.15.112:0/2075437265 learned_addr learned my addr 172.21.15.112:0/2075437265 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/2075437265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 0x55a67bf73b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 -- 172.21.15.112:0/2075437265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 msgr2=0x55a67bf73b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/2075437265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 0x55a67bf73b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 -- 172.21.15.112:0/2075437265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf67d00 msgr2=0x55a67bf5ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/2075437265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf67d00 0x55a67bf5ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 -- 172.21.15.112:0/2075437265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67bb95170 con 0x55a67bf621a0 2024-04-24T23:14:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/2075437265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6cac0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8005d00 tx=0x7fc5d80126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893878738ff8442f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5d37fe640 1 -- 172.21.15.112:0/2075437265 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d802c040 con 0x55a67bf621a0 2024-04-24T23:14:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.731+0000 7fc5d37fe640 1 -- 172.21.15.112:0/2075437265 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5d8005520 con 0x55a67bf621a0 2024-04-24T23:14:55.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5d37fe640 1 -- 172.21.15.112:0/2075437265 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8000e60 con 0x55a67bf621a0 2024-04-24T23:14:55.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 -- 172.21.15.112:0/2075437265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 msgr2=0x55a67bf6cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/2075437265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6cac0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8005d00 tx=0x7fc5d80126f0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 -- 172.21.15.112:0/2075437265 shutdown_connections 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/2075437265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf67d00 0x55a67bf5ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/2075437265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf62b90 0x55a67bf73b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/2075437265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf621a0 0x55a67bf6cac0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 -- 172.21.15.112:0/2075437265 >> 172.21.15.112:0/2075437265 conn(0x55a67bf25950 msgr2=0x55a67bf5c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 -- 172.21.15.112:0/2075437265 shutdown_connections 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.732+0000 7fc5f29a0740 1 -- 172.21.15.112:0/2075437265 wait complete. 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 1 -- start start 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 1 Processor -- start 2024-04-24T23:14:55.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf71830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 0x55a67bf71d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf72d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81a97a0 con 0x55a67bf67d00 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81a90d0 con 0x55a67bf62b90 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e81a6310 con 0x55a67bf621a0 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5f29a0740 10 client.0 _ll_register_callbacks cb 0x7fc5d800be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf72d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf72d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39932/0 (socket says 172.21.15.112:39932) 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 -- 172.21.15.112:0/3875548527 learned_addr learned my addr 172.21.15.112:0/3875548527 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 0x55a67bf71d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf71830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 -- 172.21.15.112:0/3875548527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 msgr2=0x55a67bf71830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf71830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 -- 172.21.15.112:0/3875548527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 msgr2=0x55a67bf71d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 0x55a67bf71d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.733+0000 7fc5ed14b640 1 -- 172.21.15.112:0/3875548527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67bf73300 con 0x55a67bf67d00 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5ed14b640 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf72d10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800b950 tx=0x7fc5d8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8926808c59e8c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5d17fa640 1 -- 172.21.15.112:0/3875548527 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8016040 con 0x55a67bf67d00 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 1 -- 172.21.15.112:0/3875548527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5d8005520 con 0x55a67bf67d00 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5d17fa640 1 -- 172.21.15.112:0/3875548527 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc5d8005740 con 0x55a67bf67d00 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5d17fa640 1 -- 172.21.15.112:0/3875548527 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5d804b050 con 0x55a67bf67d00 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 10 client.15675 fetch_fsmap learned FSMap version 28 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 1 -- 172.21.15.112:0/3875548527 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a67bfde9d0 con 0x55a67bf67d00 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5d17fa640 1 -- 172.21.15.112:0/3875548527 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5d8005520 con 0x55a67bf67d00 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5d17fa640 1 -- 172.21.15.112:0/3875548527 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc5d8026070 con 0x55a67bf67d00 2024-04-24T23:14:55.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 10 client.15675 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 20 client.15675 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55a67bfdf2c0 0x55a67bfe16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 4 client.15675 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.734+0000 7fc5f29a0740 1 -- 172.21.15.112:0/3875548527 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a67bece800 con 0x55a67bfdf2c0 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.735+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55a67bfdf2c0 0x55a67bfe16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.735+0000 7fc5d17fa640 10 client.15675 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.735+0000 7fc5ed94c640 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55a67bfdf2c0 0x55a67bfe16e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc4b215e2028fa server_cookie=6c72f7e1ae15c32b in_seq=0 out_seq=0 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.736+0000 7fc5d17fa640 1 -- 172.21.15.112:0/3875548527 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a67bece800 con 0x55a67bfdf2c0 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.736+0000 7fc5d17fa640 10 client.15675 handle_command_reply: tid=0 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.736+0000 7fc5f29a0740 1 client.15675 shutdown 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.736+0000 7fc5f29a0740 1 -- 172.21.15.112:0/3875548527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 msgr2=0x55a67bf72d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.736+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf72d10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800b950 tx=0x7fc5d8021040 comp rx=0 tx=0).stop 2024-04-24T23:14:55.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 1 -- 172.21.15.112:0/3875548527 shutdown_connections 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55a67bfdf2c0 0x55a67bfe16e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a67bf67d00 0x55a67bf72d10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5d17fa640 0 client.15675 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a67bf62b90 0x55a67bf71d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 1 --2- 172.21.15.112:0/3875548527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a67bf621a0 0x55a67bf71830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 1 -- 172.21.15.112:0/3875548527 >> 172.21.15.112:0/3875548527 conn(0x55a67bf25950 msgr2=0x55a67bf6c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 1 -- 172.21.15.112:0/3875548527 shutdown_connections 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 1 -- 172.21.15.112:0/3875548527 wait complete. 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5f29a0740 20 client.15675 trim_cache size 0 max 16384 2024-04-24T23:14:55.744 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:55.745 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-04-24T23:14:55.745 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "423d772a-a433-48ab-81ad-fd38a5b10d85", 2024-04-24T23:14:55.745 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-04-24T23:14:55.745 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 -- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5d40a2b90 msgr2=0x7fc5d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5d40a2b90 0x7fc5d40a5050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00b9ad0 tx=0x7fc5e008f110 comp rx=0 tx=0).stop 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 -- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 msgr2=0x7fc5e819f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 0x7fc5e819f0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0009df0 tx=0x7fc5e0003ba0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 -- 172.21.15.112:0/3588887979 shutdown_connections 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5d40a2b90 0x7fc5d40a5050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc5e81033e0 0x7fc5e819f0a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5e8102a20 0x7fc5e8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 --2- 172.21.15.112:0/3588887979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5e80ffa50 0x7fc5e8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 -- 172.21.15.112:0/3588887979 >> 172.21.15.112:0/3588887979 conn(0x7fc5e80fb380 msgr2=0x7fc5e8105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 -- 172.21.15.112:0/3588887979 shutdown_connections 2024-04-24T23:14:55.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.737+0000 7fc5efbb0640 1 -- 172.21.15.112:0/3588887979 wait complete. 2024-04-24T23:14:55.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:14:55.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:14:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.888+0000 7f72b8de2640 1 -- 172.21.15.112:0/3330356832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4102230 msgr2=0x7f72b4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.888+0000 7f72b8de2640 1 --2- 172.21.15.112:0/3330356832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4102230 0x7f72b4108ac0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f72ac009900 tx=0x7f72ac02a410 comp rx=0 tx=0).stop 2024-04-24T23:14:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 -- 172.21.15.112:0/3330356832 shutdown_connections 2024-04-24T23:14:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 --2- 172.21.15.112:0/3330356832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4102230 0x7f72b4108ac0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 --2- 172.21.15.112:0/3330356832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b4101870 0x7f72b4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 --2- 172.21.15.112:0/3330356832 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b4100680 0x7f72b4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 -- 172.21.15.112:0/3330356832 >> 172.21.15.112:0/3330356832 conn(0x7f72b40fbbf0 msgr2=0x7f72b40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 -- 172.21.15.112:0/3330356832 shutdown_connections 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 -- 172.21.15.112:0/3330356832 wait complete. 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 Processor -- start 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.889+0000 7f72b8de2640 1 -- start start 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b8de2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4100680 0x7f72b4197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b8de2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b4101870 0x7f72b4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b8de2640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b4102230 0x7f72b419f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b8de2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72b410c430 con 0x7f72b4100680 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b8de2640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f72b410c2b0 con 0x7f72b4102230 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b8de2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72b410c5b0 con 0x7f72b4101870 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4100680 0x7f72b4197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4100680 0x7f72b4197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39954/0 (socket says 172.21.15.112:39954) 2024-04-24T23:14:55.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 -- 172.21.15.112:0/2819780480 learned_addr learned my addr 172.21.15.112:0/2819780480 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b4101870 0x7f72b4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b4102230 0x7f72b419f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 -- 172.21.15.112:0/2819780480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b4101870 msgr2=0x7f72b4198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b4101870 0x7f72b4198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 -- 172.21.15.112:0/2819780480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b4102230 msgr2=0x7f72b419f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b4102230 0x7f72b419f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 -- 172.21.15.112:0/2819780480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b419f760 con 0x7f72b4100680 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b4102230 0x7f72b419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b4101870 0x7f72b4198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b259c640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4100680 0x7f72b4197b40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f72a400ccb0 tx=0x7f72a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72a37fe640 1 -- 172.21.15.112:0/2819780480 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4003990 con 0x7f72b4100680 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72a37fe640 1 -- 172.21.15.112:0/2819780480 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72a4004520 con 0x7f72b4100680 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b8de2640 1 -- 172.21.15.112:0/2819780480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b419fa50 con 0x7f72b4100680 2024-04-24T23:14:55.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72a37fe640 1 -- 172.21.15.112:0/2819780480 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a400f870 con 0x7f72b4100680 2024-04-24T23:14:55.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.890+0000 7f72b8de2640 1 -- 172.21.15.112:0/2819780480 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b419ffa0 con 0x7f72b4100680 2024-04-24T23:14:55.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.892+0000 7f72a37fe640 1 -- 172.21.15.112:0/2819780480 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f72a40046d0 con 0x7f72b4100680 2024-04-24T23:14:55.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72bbbc5740 1 Processor -- start 2024-04-24T23:14:55.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72bbbc5740 1 -- start start 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3e1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 0x5585df3e6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 0x5585df3e9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5585df3456b0 con 0x5585df3e1240 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5585df2ed880 con 0x5585df3e6da0 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5585df341670 con 0x5585df3e1c30 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72a37fe640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f729c0a2b90 0x7f729c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3e1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3e1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39962/0 (socket says 172.21.15.112:39962) 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 -- 172.21.15.112:0/1898948257 learned_addr learned my addr 172.21.15.112:0/1898948257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f729c0a2b90 0x7f729c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b259c640 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 0x5585df3e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 0x5585df3e9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72a37fe640 1 -- 172.21.15.112:0/2819780480 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f72a40cf8b0 con 0x7f72b4100680 2024-04-24T23:14:55.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 -- 172.21.15.112:0/1898948257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 msgr2=0x5585df3e6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 0x5585df3e6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 -- 172.21.15.112:0/1898948257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 msgr2=0x5585df3e9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 0x5585df3e9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.893+0000 7f72b2d9d640 1 -- 172.21.15.112:0/1898948257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585df1237d0 con 0x5585df3e1240 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.894+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3e1660 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f72ac031be0 tx=0x7f72ac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c361daf68d58c7a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.894+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f729c0a2b90 0x7f729c0a5050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72ac02a930 tx=0x7f72ac045000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.894+0000 7f72a0ff9640 1 -- 172.21.15.112:0/1898948257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac035040 con 0x5585df3e1240 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.894+0000 7f72a0ff9640 1 -- 172.21.15.112:0/1898948257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72ac0043e0 con 0x5585df3e1240 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.894+0000 7f72a0ff9640 1 -- 172.21.15.112:0/1898948257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac048630 con 0x5585df3e1240 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- 172.21.15.112:0/1898948257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 msgr2=0x5585df3e1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3e1660 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f72ac031be0 tx=0x7f72ac004040 comp rx=0 tx=0).stop 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- 172.21.15.112:0/1898948257 shutdown_connections 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 0x5585df3e9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 0x5585df3e6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/1898948257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3e1660 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- 172.21.15.112:0/1898948257 >> 172.21.15.112:0/1898948257 conn(0x5585df3dc8b0 msgr2=0x5585df3dccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- 172.21.15.112:0/1898948257 shutdown_connections 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- 172.21.15.112:0/1898948257 wait complete. 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- start start 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 Processor -- start 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1c30 0x5585df3ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e6da0 0x5585df3f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5585df3e9cf0 con 0x5585df3e1240 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5585df300f50 con 0x5585df3e1c30 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5585df3e9e70 con 0x5585df3e6da0 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72bbbc5740 10 client.0 _ll_register_callbacks cb 0x5585df33c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:55.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72b259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3eb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39964/0 (socket says 172.21.15.112:39964) 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72b259c640 1 -- 172.21.15.112:0/2160644591 learned_addr learned my addr 172.21.15.112:0/2160644591 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.895+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1c30 0x5585df3ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e6da0 0x5585df3f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b259c640 1 -- 172.21.15.112:0/2160644591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e6da0 msgr2=0x5585df3f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b259c640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e6da0 0x5585df3f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b259c640 1 -- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1c30 msgr2=0x5585df3ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b259c640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1c30 0x5585df3ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b259c640 1 -- 172.21.15.112:0/2160644591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585df0593f0 con 0x5585df3e1240 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e6da0 0x5585df3f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1c30 0x5585df3ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72b259c640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3eb630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f72a409ba20 tx=0x7f72a4066d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad718fff3faf66e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f7299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:55.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f7299ffb640 1 -- 172.21.15.112:0/2160644591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a401d040 con 0x5585df3e1240 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f7299ffb640 1 -- 172.21.15.112:0/2160644591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f72a4066960 con 0x5585df3e1240 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2160644591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ac0043e0 con 0x5585df3e1240 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f7299ffb640 1 -- 172.21.15.112:0/2160644591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72a4066e00 con 0x5585df3e1240 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 10 client.15687 fetch_fsmap learned FSMap version 28 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2160644591 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5585df0190f0 con 0x5585df3e1240 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f7299ffb640 1 -- 172.21.15.112:0/2160644591 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72ac0043e0 con 0x5585df3e1240 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f7299ffb640 1 -- 172.21.15.112:0/2160644591 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72a4063070 con 0x5585df3e1240 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 10 client.15687 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 20 client.15687 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5585df3f7540 0x5585df3f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 4 client.15687 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.896+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2160644591 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585df352bf0 con 0x5585df3f7540 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.897+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5585df3f7540 0x5585df3f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.897+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5585df3f7540 0x5585df3f99b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e789171b14773e70 server_cookie=f1ef28f8d11d0d47 in_seq=0 out_seq=0 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.897+0000 7f7299ffb640 10 client.15687 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f7299ffb640 1 -- 172.21.15.112:0/2160644591 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585df352bf0 con 0x5585df3f7540 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f7299ffb640 10 client.15687 handle_command_reply: tid=0 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 client.15687 shutdown 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2160644591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 msgr2=0x5585df3eb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3eb630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f72a409ba20 tx=0x7f72a4066d90 comp rx=0 tx=0).stop 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2160644591 shutdown_connections 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5585df3f7540 0x5585df3f99b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e6da0 0x5585df3f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f7299ffb640 0 client.15687 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1c30 0x5585df3ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2160644591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1240 0x5585df3eb630 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2160644591 >> 172.21.15.112:0/2160644591 conn(0x5585df3dc8b0 msgr2=0x5585df3dbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2160644591 shutdown_connections 2024-04-24T23:14:55.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2160644591 wait complete. 2024-04-24T23:14:55.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.898+0000 7f72bbbc5740 20 client.15687 trim_cache size 0 max 16384 2024-04-24T23:14:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.915+0000 7f72bbbc5740 1 Processor -- start 2024-04-24T23:14:55.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.915+0000 7f72bbbc5740 1 -- start start 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1240 0x5585df3e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 0x5585df3f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e6da0 0x5585df3eb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41aa620 con 0x5585df3e6da0 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41ac160 con 0x5585df3e1240 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41aacf0 con 0x5585df3e1c30 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e6da0 0x5585df3eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e6da0 0x5585df3eb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39978/0 (socket says 172.21.15.112:39978) 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 -- 172.21.15.112:0/2238265380 learned_addr learned my addr 172.21.15.112:0/2238265380 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/2238265380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 0x5585df3f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 -- 172.21.15.112:0/2238265380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 msgr2=0x5585df3f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2238265380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 0x5585df3f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 -- 172.21.15.112:0/2238265380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1240 msgr2=0x5585df3e24c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2238265380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1240 0x5585df3e24c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 -- 172.21.15.112:0/2238265380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585df3e3740 con 0x5585df3e6da0 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f72b2d9d640 1 --2- 172.21.15.112:0/2238265380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e6da0 0x5585df3eb950 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0040b0 tx=0x7f72ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7066f87a2eebfcce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f729bfff640 1 -- 172.21.15.112:0/2238265380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac035040 con 0x5585df3e6da0 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f729bfff640 1 -- 172.21.15.112:0/2238265380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72ac032d00 con 0x5585df3e6da0 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.916+0000 7f729bfff640 1 -- 172.21.15.112:0/2238265380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0488a0 con 0x5585df3e6da0 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2238265380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e6da0 msgr2=0x5585df3eb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2238265380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e6da0 0x5585df3eb950 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0040b0 tx=0x7f72ac009260 comp rx=0 tx=0).stop 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2238265380 shutdown_connections 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2238265380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e6da0 0x5585df3eb950 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2238265380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1c30 0x5585df3f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/2238265380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e1240 0x5585df3e24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2238265380 >> 172.21.15.112:0/2238265380 conn(0x5585df3dc8b0 msgr2=0x5585df3eb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2238265380 shutdown_connections 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 -- 172.21.15.112:0/2238265380 wait complete. 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 -- start start 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.917+0000 7f72bbbc5740 1 Processor -- start 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1240 0x5585df3efc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1c30 0x5585df3f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 0x5585df45ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41aba90 con 0x5585df3e1c30 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41ab3c0 con 0x5585df3e6da0 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72b41a8600 con 0x5585df3e1240 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 10 client.0 _ll_register_callbacks cb 0x7f72a4097fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1c30 0x5585df3f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1c30 0x5585df3f01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39990/0 (socket says 172.21.15.112:39990) 2024-04-24T23:14:55.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 -- 172.21.15.112:0/3801533071 learned_addr learned my addr 172.21.15.112:0/3801533071 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b259c640 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1240 0x5585df3efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 -- 172.21.15.112:0/3801533071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1240 msgr2=0x5585df3efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1240 0x5585df3efc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 -- 172.21.15.112:0/3801533071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 msgr2=0x5585df45ba90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 0x5585df45ba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 -- 172.21.15.112:0/3801533071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585df45c080 con 0x5585df3e1c30 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b259c640 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1240 0x5585df3efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72b1d9b640 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1c30 0x5585df3f01d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5585df3dd820 tx=0x7f72a80128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac41219a9f08840a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f7299ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f7299ffb640 1 -- 172.21.15.112:0/3801533071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8022070 con 0x5585df3e1c30 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f7299ffb640 1 -- 172.21.15.112:0/3801533071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f72a801e850 con 0x5585df3e1c30 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 1 -- 172.21.15.112:0/3801533071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72ac032d00 con 0x5585df3e1c30 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f7299ffb640 1 -- 172.21.15.112:0/3801533071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f72a802e050 con 0x5585df3e1c30 2024-04-24T23:14:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 10 client.15693 fetch_fsmap learned FSMap version 28 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f72bbbc5740 1 -- 172.21.15.112:0/3801533071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5585df45d490 con 0x5585df3e1c30 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.918+0000 7f7299ffb640 1 -- 172.21.15.112:0/3801533071 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f72ac032d00 con 0x5585df3e1c30 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.919+0000 7f7299ffb640 1 -- 172.21.15.112:0/3801533071 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f72a8026410 con 0x5585df3e1c30 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.919+0000 7f72bbbc5740 10 client.15693 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.919+0000 7f72bbbc5740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.919+0000 7f72bbbc5740 20 client.15693 populate_metadata read hostname 'smithi112' 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.919+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5585df45dec0 0x5585df4602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.919+0000 7f72bbbc5740 4 client.15693 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:14:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.919+0000 7f72bbbc5740 1 -- 172.21.15.112:0/3801533071 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585df34cf40 con 0x5585df45dec0 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.921+0000 7f72b259c640 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5585df45dec0 0x5585df4602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.922+0000 7f7299ffb640 10 client.15693 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.922+0000 7f72b259c640 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5585df45dec0 0x5585df4602e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78155d5ca4f5574e server_cookie=d9e1f8f61dbb8abd in_seq=0 out_seq=0 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.922+0000 7f7299ffb640 1 -- 172.21.15.112:0/3801533071 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5585df34cf40 con 0x5585df45dec0 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f7299ffb640 10 client.15693 handle_command_reply: tid=0 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 client.15693 shutdown 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 -- 172.21.15.112:0/3801533071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1c30 msgr2=0x5585df3f01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1c30 0x5585df3f01d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5585df3dd820 tx=0x7f72a80128e0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 -- 172.21.15.112:0/3801533071 shutdown_connections 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5585df45dec0 0x5585df4602e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5585df3e6da0 0x5585df45ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f7299ffb640 0 client.15693 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5585df3e1c30 0x5585df3f01d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 --2- 172.21.15.112:0/3801533071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585df3e1240 0x5585df3efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 -- 172.21.15.112:0/3801533071 >> 172.21.15.112:0/3801533071 conn(0x5585df3dc8b0 msgr2=0x5585df3db260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 -- 172.21.15.112:0/3801533071 shutdown_connections 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 1 -- 172.21.15.112:0/3801533071 wait complete. 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.923+0000 7f72bbbc5740 20 client.15693 trim_cache size 0 max 16384 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-04-24T23:14:55.929 INFO:teuthology.orchestra.run.smithi112.stdout: "423d772a-a433-48ab-81ad-fd38a5b10d85": { 2024-04-24T23:14:55.930 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-04-24T23:14:55.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "423d772a-a433-48ab-81ad-fd38a5b10d85", 2024-04-24T23:14:55.930 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-04-24T23:14:55.930 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:14:55.930 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:14:55.930 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 -- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f729c0a2b90 msgr2=0x7f729c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f729c0a2b90 0x7f729c0a5050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72ac02a930 tx=0x7f72ac045000 comp rx=0 tx=0).stop 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 -- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4100680 msgr2=0x7f72b4197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4100680 0x7f72b4197b40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f72a400ccb0 tx=0x7f72a400a620 comp rx=0 tx=0).stop 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 -- 172.21.15.112:0/2819780480 shutdown_connections 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f729c0a2b90 0x7f729c0a5050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72b4102230 0x7f72b419f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b4101870 0x7f72b4198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 --2- 172.21.15.112:0/2819780480 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b4100680 0x7f72b4197b40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 -- 172.21.15.112:0/2819780480 >> 172.21.15.112:0/2819780480 conn(0x7f72b40fbbf0 msgr2=0x7f72b40fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 -- 172.21.15.112:0/2819780480 shutdown_connections 2024-04-24T23:14:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:14:55.924+0000 7f72b8de2640 1 -- 172.21.15.112:0/2819780480 wait complete. 2024-04-24T23:14:55.937 INFO:tasks.cephfs.filesystem:scrub status for tag:423d772a-a433-48ab-81ad-fd38a5b10d85 - {'path': '/', 'tag': '423d772a-a433-48ab-81ad-fd38a5b10d85', 'options': 'recursive,force'} 2024-04-24T23:14:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:56 smithi112 ceph-mon[29186]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-24T23:14:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:56 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/595080901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:56 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2907853041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:56 smithi150 ceph-mon[32247]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-24T23:14:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:56 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/595080901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:56 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2907853041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:56.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:56 smithi187 ceph-mon[33292]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-24T23:14:56.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:56 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/595080901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:14:56.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:56 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2907853041' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:14:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:57 smithi150 ceph-mon[32247]: scrub queued for path: / 2024-04-24T23:14:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:57 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:57 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:14:57.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:57 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:57.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:57 smithi187 ceph-mon[33292]: scrub queued for path: / 2024-04-24T23:14:57.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:57 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:57.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:57 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:14:57.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:57 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:57.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:57 smithi112 ceph-mon[29186]: scrub queued for path: / 2024-04-24T23:14:57.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:57 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:57.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:57 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:14:57.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:57 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:14:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:58 smithi150 ceph-mon[32247]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-24T23:14:58.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:58 smithi187 ceph-mon[33292]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-24T23:14:58.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:58 smithi112 ceph-mon[29186]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-04-24T23:14:59.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:14:59 smithi112 ceph-mon[29186]: pgmap v370: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-24T23:14:59.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:14:59 smithi187 ceph-mon[33292]: pgmap v370: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-24T23:14:59.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:14:59 smithi150 ceph-mon[32247]: pgmap v370: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-24T23:15:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:01 smithi187 ceph-mon[33292]: pgmap v371: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-24T23:15:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:01 smithi112 ceph-mon[29186]: pgmap v371: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-24T23:15:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:01 smithi150 ceph-mon[32247]: pgmap v371: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-04-24T23:15:04.120 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:03 smithi187 ceph-mon[33292]: pgmap v372: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-24T23:15:04.120 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:03 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:04.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:03 smithi112 ceph-mon[29186]: pgmap v372: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-24T23:15:04.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:03 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:04.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:03 smithi150 ceph-mon[32247]: pgmap v372: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-24T23:15:04.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:03 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: pgmap v373: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:06.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:06.030 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: pgmap v373: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-24T23:15:06.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:06.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:06.069 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:06.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: pgmap v373: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-24T23:15:06.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:06.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:06.238 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:15:07.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:06 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:15:07.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:06 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:15:07.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:06 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:15:08.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:07 smithi112 ceph-mon[29186]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-24T23:15:08.068 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:07 smithi150 ceph-mon[32247]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-24T23:15:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:07 smithi187 ceph-mon[33292]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-24T23:15:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:09 smithi187 ceph-mon[33292]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-24T23:15:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:09 smithi112 ceph-mon[29186]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-24T23:15:10.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:09 smithi150 ceph-mon[32247]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-04-24T23:15:12.227 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:11 smithi187 ceph-mon[33292]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T23:15:12.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:11 smithi112 ceph-mon[29186]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T23:15:12.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:11 smithi150 ceph-mon[32247]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T23:15:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:13 smithi187 ceph-mon[33292]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:14.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:13 smithi112 ceph-mon[29186]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:13 smithi150 ceph-mon[32247]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:15 smithi187 ceph-mon[33292]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:15 smithi112 ceph-mon[29186]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:15 smithi150 ceph-mon[32247]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:17 smithi187 ceph-mon[33292]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:17 smithi112 ceph-mon[29186]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:17 smithi150 ceph-mon[32247]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:19 smithi187 ceph-mon[33292]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:20.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:19 smithi112 ceph-mon[29186]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:19 smithi150 ceph-mon[32247]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:21.986 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:21 smithi187 ceph-mon[33292]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:22.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:21 smithi112 ceph-mon[29186]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:21 smithi150 ceph-mon[32247]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:23 smithi187 ceph-mon[33292]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:24.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:23 smithi112 ceph-mon[29186]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:23 smithi150 ceph-mon[32247]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:25.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:15:25.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:15:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 -- 172.21.15.112:0/3517122824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128103330 msgr2=0x7fc128109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/3517122824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128103330 0x7fc128109bc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc120009ca0 tx=0x7fc12002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 -- 172.21.15.112:0/3517122824 shutdown_connections 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/3517122824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128103330 0x7fc128109bc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/3517122824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc128102970 0x7fc128102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/3517122824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc128101780 0x7fc128101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 -- 172.21.15.112:0/3517122824 >> 172.21.15.112:0/3517122824 conn(0x7fc1280fccf0 msgr2=0x7fc1280ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 -- 172.21.15.112:0/3517122824 shutdown_connections 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 -- 172.21.15.112:0/3517122824 wait complete. 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 Processor -- start 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.042+0000 7fc12f4ca640 1 -- start start 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12f4ca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128101780 0x7fc128197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12f4ca640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc128102970 0x7fc128198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12f4ca640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc128103330 0x7fc12819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12f4ca640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc12810d530 con 0x7fc128101780 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12f4ca640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc12810d3b0 con 0x7fc128102970 2024-04-24T23:15:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12f4ca640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc12810d6b0 con 0x7fc128103330 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128101780 0x7fc128197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128101780 0x7fc128197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52534/0 (socket says 172.21.15.112:52534) 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 -- 172.21.15.112:0/631242755 learned_addr learned my addr 172.21.15.112:0/631242755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12ca65640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc128102970 0x7fc128198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12da67640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc128103330 0x7fc12819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 -- 172.21.15.112:0/631242755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc128103330 msgr2=0x7fc12819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc128103330 0x7fc12819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 -- 172.21.15.112:0/631242755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc128102970 msgr2=0x7fc128198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc128102970 0x7fc128198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 -- 172.21.15.112:0/631242755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12819f770 con 0x7fc128101780 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12da67640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc128103330 0x7fc12819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12d266640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128101780 0x7fc128197b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc12400d950 tx=0x7fc12400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc11e7fc640 1 -- 172.21.15.112:0/631242755 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124017070 con 0x7fc128101780 2024-04-24T23:15:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12f4ca640 1 -- 172.21.15.112:0/631242755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12819fa60 con 0x7fc128101780 2024-04-24T23:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc11e7fc640 1 -- 172.21.15.112:0/631242755 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc124009c30 con 0x7fc128101780 2024-04-24T23:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc11e7fc640 1 -- 172.21.15.112:0/631242755 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12400f6b0 con 0x7fc128101780 2024-04-24T23:15:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.043+0000 7fc12f4ca640 1 -- 172.21.15.112:0/631242755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12819ffb0 con 0x7fc128101780 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc11e7fc640 1 -- 172.21.15.112:0/631242755 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc12401a070 con 0x7fc128101780 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc1321d7740 1 Processor -- start 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc1321d7740 1 -- start start 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed8c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 0x56262ed91730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed94230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc1321d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56262ecf06b0 con 0x56262ed91da0 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc1321d7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56262ec98880 con 0x56262ed8c240 2024-04-24T23:15:26.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc1321d7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56262ecec670 con 0x56262ed8cc30 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc12ca65640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed94230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc12ca65640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed94230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52542/0 (socket says 172.21.15.112:52542) 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc12ca65640 1 -- 172.21.15.112:0/2450767775 learned_addr learned my addr 172.21.15.112:0/2450767775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc12da67640 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed8c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.046+0000 7fc12d266640 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 0x56262ed91730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc11e7fc640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2b90 0x7fc1140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc12ca65640 1 -- 172.21.15.112:0/2450767775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 msgr2=0x56262ed91730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 0x56262ed91730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc12ca65640 1 -- 172.21.15.112:0/2450767775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 msgr2=0x56262ed8c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed8c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc12ca65640 1 -- 172.21.15.112:0/2450767775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262eace7d0 con 0x56262ed91da0 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc12da67640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2b90 0x7fc1140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc11e7fc640 1 -- 172.21.15.112:0/631242755 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc1240ceed0 con 0x7fc128101780 2024-04-24T23:15:26.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed94230 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc11800bcb0 tx=0x7fc1180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e988bcaafa15cc1d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc1137fe640 1 -- 172.21.15.112:0/2450767775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118004030 con 0x56262ed91da0 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc1137fe640 1 -- 172.21.15.112:0/2450767775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc1180041d0 con 0x56262ed91da0 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.047+0000 7fc1137fe640 1 -- 172.21.15.112:0/2450767775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180118a0 con 0x56262ed91da0 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 -- 172.21.15.112:0/2450767775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 msgr2=0x56262ed94230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc12da67640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2b90 0x7fc1140a5050 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc12002ade0 tx=0x7fc1200095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed94230 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc11800bcb0 tx=0x7fc1180107a0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 -- 172.21.15.112:0/2450767775 shutdown_connections 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed94230 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 0x56262ed91730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2450767775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed8c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 -- 172.21.15.112:0/2450767775 >> 172.21.15.112:0/2450767775 conn(0x56262ed878b0 msgr2=0x56262ed87cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 -- 172.21.15.112:0/2450767775 shutdown_connections 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 -- 172.21.15.112:0/2450767775 wait complete. 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 -- start start 2024-04-24T23:15:26.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.048+0000 7fc1321d7740 1 Processor -- start 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8c240 0x56262ed966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8cc30 0x56262ed96c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed9dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc1321d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56262ed94d50 con 0x56262ed8c240 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc1321d7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56262ecabf50 con 0x56262ed8cc30 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc1321d7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56262ed94ed0 con 0x56262ed91da0 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc1321d7740 10 client.0 _ll_register_callbacks cb 0x56262ece7000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8c240 0x56262ed966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8c240 0x56262ed966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52552/0 (socket says 172.21.15.112:52552) 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 -- 172.21.15.112:0/2120132721 learned_addr learned my addr 172.21.15.112:0/2120132721 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12da67640 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed9dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 -- 172.21.15.112:0/2120132721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 msgr2=0x56262ed9dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8cc30 0x56262ed96c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed9dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 -- 172.21.15.112:0/2120132721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8cc30 msgr2=0x56262ed96c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8cc30 0x56262ed96c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 -- 172.21.15.112:0/2120132721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262ea043f0 con 0x56262ed8c240 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc12d266640 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8c240 0x56262ed966d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc1240cc5e0 tx=0x7fc12400d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca9cbdeac69711c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc110ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc110ff9640 1 -- 172.21.15.112:0/2120132721 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124021070 con 0x56262ed8c240 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc110ff9640 1 -- 172.21.15.112:0/2120132721 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc1240109f0 con 0x56262ed8c240 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.049+0000 7fc1321d7740 1 -- 172.21.15.112:0/2120132721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1180041d0 con 0x56262ed8c240 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc110ff9640 1 -- 172.21.15.112:0/2120132721 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc12409aca0 con 0x56262ed8c240 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc1321d7740 10 client.15705 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc1321d7740 1 -- 172.21.15.112:0/2120132721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56262e9c40f0 con 0x56262ed8c240 2024-04-24T23:15:26.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc110ff9640 1 -- 172.21.15.112:0/2120132721 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc1180041d0 con 0x56262ed8c240 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc110ff9640 1 -- 172.21.15.112:0/2120132721 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc1240d2b60 con 0x56262ed8c240 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc1321d7740 10 client.15705 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc1321d7740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc1321d7740 20 client.15705 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56262eda2620 0x56262eda4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc1321d7740 4 client.15705 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc1321d7740 1 -- 172.21.15.112:0/2120132721 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56262ecfdbf0 con 0x56262eda2620 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.050+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56262eda2620 0x56262eda4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56262eda2620 0x56262eda4ac0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1285d685d776e963 server_cookie=343fa9372dfec7ca in_seq=0 out_seq=0 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc110ff9640 10 client.15705 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc110ff9640 1 -- 172.21.15.112:0/2120132721 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56262ecfdbf0 con 0x56262eda2620 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc110ff9640 10 client.15705 handle_command_reply: tid=0 2024-04-24T23:15:26.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 client.15705 shutdown 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 -- 172.21.15.112:0/2120132721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8c240 msgr2=0x56262ed966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8c240 0x56262ed966d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc1240cc5e0 tx=0x7fc12400d8d0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 -- 172.21.15.112:0/2120132721 shutdown_connections 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56262eda2620 0x56262eda4ac0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed9dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc110ff9640 0 client.15705 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8cc30 0x56262ed96c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2120132721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8c240 0x56262ed966d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.051+0000 7fc1321d7740 1 -- 172.21.15.112:0/2120132721 >> 172.21.15.112:0/2120132721 conn(0x56262ed878b0 msgr2=0x56262ed86af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.052+0000 7fc1321d7740 1 -- 172.21.15.112:0/2120132721 shutdown_connections 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.052+0000 7fc1321d7740 1 -- 172.21.15.112:0/2120132721 wait complete. 2024-04-24T23:15:26.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.052+0000 7fc1321d7740 20 client.15705 trim_cache size 0 max 16384 2024-04-24T23:15:26.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc1321d7740 1 Processor -- start 2024-04-24T23:15:26.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc1321d7740 1 -- start start 2024-04-24T23:15:26.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed96b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8cc30 0x56262ed9dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed87c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc1321d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281aa580 con 0x56262ed8cc30 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc1321d7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281ac0c0 con 0x56262ed8c240 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc1321d7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281aac50 con 0x56262ed91da0 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc12ca65640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8cc30 0x56262ed9dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc12ca65640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8cc30 0x56262ed9dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52554/0 (socket says 172.21.15.112:52554) 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc12ca65640 1 -- 172.21.15.112:0/2654776935 learned_addr learned my addr 172.21.15.112:0/2654776935 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc12ca65640 1 -- 172.21.15.112:0/2654776935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 msgr2=0x56262ed87c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.069+0000 7fc12da67640 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed87c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc12d266640 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed96b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed87c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc12ca65640 1 -- 172.21.15.112:0/2654776935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 msgr2=0x56262ed96b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed96b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc12ca65640 1 -- 172.21.15.112:0/2654776935 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262ed79090 con 0x56262ed8cc30 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc12da67640 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed87c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc12ca65640 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8cc30 0x56262ed9dc10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc118000c00 tx=0x7fc11800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27366e92308c52e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc112ffd640 1 -- 172.21.15.112:0/2654776935 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118028470 con 0x56262ed8cc30 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc112ffd640 1 -- 172.21.15.112:0/2654776935 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc118005540 con 0x56262ed8cc30 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.070+0000 7fc112ffd640 1 -- 172.21.15.112:0/2654776935 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118003a50 con 0x56262ed8cc30 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- 172.21.15.112:0/2654776935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8cc30 msgr2=0x56262ed9dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8cc30 0x56262ed9dc10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc118000c00 tx=0x7fc11800bf30 comp rx=0 tx=0).stop 2024-04-24T23:15:26.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- 172.21.15.112:0/2654776935 shutdown_connections 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed91da0 0x56262ed87c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed8cc30 0x56262ed9dc10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 --2- 172.21.15.112:0/2654776935 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed96b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- 172.21.15.112:0/2654776935 >> 172.21.15.112:0/2654776935 conn(0x56262ed878b0 msgr2=0x56262ed86260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- 172.21.15.112:0/2654776935 shutdown_connections 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- 172.21.15.112:0/2654776935 wait complete. 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- start start 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 Processor -- start 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed9b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 0x56262ed9b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed9ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281ab9f0 con 0x56262ed91da0 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281ab320 con 0x56262ed8c240 2024-04-24T23:15:26.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc1281a8560 con 0x56262ed8cc30 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc1321d7740 10 client.0 _ll_register_callbacks cb 0x7fc12400bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.071+0000 7fc12da67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed9ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12da67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed9ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52568/0 (socket says 172.21.15.112:52568) 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12da67640 1 -- 172.21.15.112:0/128423437 learned_addr learned my addr 172.21.15.112:0/128423437 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12d266640 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed9b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12ca65640 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 0x56262ed9b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12da67640 1 -- 172.21.15.112:0/128423437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 msgr2=0x56262ed9b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12da67640 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 0x56262ed9b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12da67640 1 -- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 msgr2=0x56262ed9b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12da67640 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed9b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12da67640 1 -- 172.21.15.112:0/128423437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56262ed9cff0 con 0x56262ed91da0 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12d266640 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:26.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc12da67640 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed9ca00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc120031f30 tx=0x7fc12002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc221bc2564bc69 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc110ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc110ff9640 1 -- 172.21.15.112:0/128423437 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1200044c0 con 0x56262ed91da0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc110ff9640 1 -- 172.21.15.112:0/128423437 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc120004660 con 0x56262ed91da0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc1321d7740 1 -- 172.21.15.112:0/128423437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc118005540 con 0x56262ed91da0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc110ff9640 1 -- 172.21.15.112:0/128423437 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc120004860 con 0x56262ed91da0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc1321d7740 10 client.15711 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.072+0000 7fc1321d7740 1 -- 172.21.15.112:0/128423437 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56262ee08d60 con 0x56262ed91da0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc110ff9640 1 -- 172.21.15.112:0/128423437 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc118005540 con 0x56262ed91da0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc110ff9640 1 -- 172.21.15.112:0/128423437 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc120030930 con 0x56262ed91da0 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc1321d7740 10 client.15711 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc1321d7740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc1321d7740 20 client.15711 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc1321d7740 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56262ee097c0 0x56262ee0bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc1321d7740 4 client.15711 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc1321d7740 1 -- 172.21.15.112:0/128423437 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56262ecf7f40 con 0x56262ee097c0 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.073+0000 7fc12d266640 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56262ee097c0 0x56262ee0bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc12d266640 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56262ee097c0 0x56262ee0bbe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4346eafdbcc0f162 server_cookie=6e2ac8f13d3321e9 in_seq=0 out_seq=0 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc110ff9640 10 client.15711 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc110ff9640 1 -- 172.21.15.112:0/128423437 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56262ecf7f40 con 0x56262ee097c0 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc110ff9640 10 client.15711 handle_command_reply: tid=0 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 client.15711 shutdown 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 -- 172.21.15.112:0/128423437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 msgr2=0x56262ed9ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed9ca00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc120031f30 tx=0x7fc12002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 -- 172.21.15.112:0/128423437 shutdown_connections 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56262ee097c0 0x56262ee0bbe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56262ed91da0 0x56262ed9ca00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56262ed8cc30 0x56262ed9b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 --2- 172.21.15.112:0/128423437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56262ed8c240 0x56262ed9b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 -- 172.21.15.112:0/128423437 >> 172.21.15.112:0/128423437 conn(0x56262ed878b0 msgr2=0x56262ed96950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc110ff9640 0 client.15711 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:26.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 -- 172.21.15.112:0/128423437 shutdown_connections 2024-04-24T23:15:26.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 1 -- 172.21.15.112:0/128423437 wait complete. 2024-04-24T23:15:26.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.074+0000 7fc1321d7740 20 client.15711 trim_cache size 0 max 16384 2024-04-24T23:15:26.083 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:15:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-04-24T23:15:26.083 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-04-24T23:15:26.083 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:15:26.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 -- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2b90 msgr2=0x7fc1140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2b90 0x7fc1140a5050 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc12002ade0 tx=0x7fc1200095c0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 -- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128101780 msgr2=0x7fc128197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128101780 0x7fc128197b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc12400d950 tx=0x7fc12400de10 comp rx=0 tx=0).stop 2024-04-24T23:15:26.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 -- 172.21.15.112:0/631242755 shutdown_connections 2024-04-24T23:15:26.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc1140a2b90 0x7fc1140a5050 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc128103330 0x7fc12819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc128102970 0x7fc128198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 --2- 172.21.15.112:0/631242755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc128101780 0x7fc128197b50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 -- 172.21.15.112:0/631242755 >> 172.21.15.112:0/631242755 conn(0x7fc1280fccf0 msgr2=0x7fc128105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 -- 172.21.15.112:0/631242755 shutdown_connections 2024-04-24T23:15:26.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.075+0000 7fc12f4ca640 1 -- 172.21.15.112:0/631242755 wait complete. 2024-04-24T23:15:26.087 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:15:26.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 -- 172.21.15.112:0/3913467820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 msgr2=0x7f5024108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 --2- 172.21.15.112:0/3913467820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 0x7f5024108ac0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f501c009ca0 tx=0x7f501c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 -- 172.21.15.112:0/3913467820 shutdown_connections 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 --2- 172.21.15.112:0/3913467820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 0x7f5024108ac0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 --2- 172.21.15.112:0/3913467820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5024101870 0x7f5024101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 --2- 172.21.15.112:0/3913467820 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5024100680 0x7f5024100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 -- 172.21.15.112:0/3913467820 >> 172.21.15.112:0/3913467820 conn(0x7f50240fbbf0 msgr2=0x7f50240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 -- 172.21.15.112:0/3913467820 shutdown_connections 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.186+0000 7f502a942640 1 -- 172.21.15.112:0/3913467820 wait complete. 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f502a942640 1 Processor -- start 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f502a942640 1 -- start start 2024-04-24T23:15:26.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f502a942640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5024100680 0x7f5024069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f502a942640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5024101870 0x7f502419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f502a942640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 0x7f502419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f502a942640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f502410c430 con 0x7f5024102230 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f502a942640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f502410c2b0 con 0x7f5024100680 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f502a942640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f502410c5b0 con 0x7f5024101870 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f5028edf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 0x7f502419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f5028edf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 0x7f502419ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52592/0 (socket says 172.21.15.112:52592) 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f5028edf640 1 -- 172.21.15.112:0/2069595346 learned_addr learned my addr 172.21.15.112:0/2069595346 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f50237fe640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5024101870 0x7f502419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.187+0000 7f5023fff640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5024100680 0x7f5024069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f5028edf640 1 -- 172.21.15.112:0/2069595346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5024101870 msgr2=0x7f502419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f5028edf640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5024101870 0x7f502419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f5028edf640 1 -- 172.21.15.112:0/2069595346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5024100680 msgr2=0x7f5024069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f5028edf640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5024100680 0x7f5024069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f5028edf640 1 -- 172.21.15.112:0/2069595346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502419f740 con 0x7f5024102230 2024-04-24T23:15:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f5023fff640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5024100680 0x7f5024069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f5028edf640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 0x7f502419ee30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f501c02f540 tx=0x7f501c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f50217fa640 1 -- 172.21.15.112:0/2069595346 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c02fae0 con 0x7f5024102230 2024-04-24T23:15:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502419f970 con 0x7f5024102230 2024-04-24T23:15:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f50217fa640 1 -- 172.21.15.112:0/2069595346 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f501c02fc80 con 0x7f5024102230 2024-04-24T23:15:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f50217fa640 1 -- 172.21.15.112:0/2069595346 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0468c0 con 0x7f5024102230 2024-04-24T23:15:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.188+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f502419ff10 con 0x7f5024102230 2024-04-24T23:15:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.189+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50241078d0 con 0x7f5024102230 2024-04-24T23:15:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.191+0000 7f50217fa640 1 -- 172.21.15.112:0/2069595346 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f501c030bc0 con 0x7f5024102230 2024-04-24T23:15:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.191+0000 7f50217fa640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f500c0a2b90 0x7f500c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.191+0000 7f50217fa640 1 -- 172.21.15.112:0/2069595346 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f501c0f0c60 con 0x7f5024102230 2024-04-24T23:15:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.191+0000 7f5023fff640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f500c0a2b90 0x7f500c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.192+0000 7f5023fff640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f500c0a2b90 0x7f500c0a5050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50241016d0 tx=0x7f501400a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.194+0000 7f50217fa640 1 -- 172.21.15.112:0/2069595346 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f501c081740 con 0x7f5024102230 2024-04-24T23:15:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:25 smithi187 ceph-mon[33292]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:26.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:25 smithi112 ceph-mon[29186]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:25 smithi150 ceph-mon[32247]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:26.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.415+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f502402d170 con 0x7f5024102230 2024-04-24T23:15:26.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.415+0000 7f50217fa640 1 -- 172.21.15.112:0/2069595346 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f501c0bd7d0 con 0x7f5024102230 2024-04-24T23:15:26.417 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:15:26.417 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:14:48.128683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24424,24425,24490,24524,24482]},"id":1}]} 2024-04-24T23:15:26.418 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-04-24T23:15:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f500c0a2b90 msgr2=0x7f500c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f500c0a2b90 0x7f500c0a5050 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f50241016d0 tx=0x7f501400a560 comp rx=0 tx=0).stop 2024-04-24T23:15:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 msgr2=0x7f502419ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 0x7f502419ee30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f501c02f540 tx=0x7f501c009890 comp rx=0 tx=0).stop 2024-04-24T23:15:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 shutdown_connections 2024-04-24T23:15:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f500c0a2b90 0x7f500c0a5050 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5024102230 0x7f502419ee30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5024101870 0x7f502419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 --2- 172.21.15.112:0/2069595346 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5024100680 0x7f5024069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 >> 172.21.15.112:0/2069595346 conn(0x7f50240fbbf0 msgr2=0x7f5024104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 shutdown_connections 2024-04-24T23:15:26.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.418+0000 7f502a942640 1 -- 172.21.15.112:0/2069595346 wait complete. 2024-04-24T23:15:26.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.532+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/2980435805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 msgr2=0x7fbeac108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.532+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/2980435805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 0x7fbeac108ac0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009ca0 tx=0x7fbea402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/2980435805 shutdown_connections 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/2980435805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 0x7fbeac108ac0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/2980435805 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeac101870 0x7fbeac101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/2980435805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac100680 0x7fbeac100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/2980435805 >> 172.21.15.112:0/2980435805 conn(0x7fbeac0fbbf0 msgr2=0x7fbeac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/2980435805 shutdown_connections 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/2980435805 wait complete. 2024-04-24T23:15:26.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 Processor -- start 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.533+0000 7fbeb0a4d640 1 -- start start 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeb0a4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac100680 0x7fbeac196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeb0a4d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeac101870 0x7fbeac196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeb0a4d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 0x7fbeac19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeb0a4d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac10c420 con 0x7fbeac102230 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeb0a4d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac10c2a0 con 0x7fbeac101870 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeb0a4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac10c5a0 con 0x7fbeac100680 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 0x7fbeac19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 0x7fbeac19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52620/0 (socket says 172.21.15.112:52620) 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 -- 172.21.15.112:0/3117764565 learned_addr learned my addr 172.21.15.112:0/3117764565 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac100680 0x7fbeac196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeac101870 0x7fbeac196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 -- 172.21.15.112:0/3117764565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac100680 msgr2=0x7fbeac196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac100680 0x7fbeac196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 -- 172.21.15.112:0/3117764565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeac101870 msgr2=0x7fbeac196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeac101870 0x7fbeac196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 -- 172.21.15.112:0/3117764565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac19e620 con 0x7fbeac102230 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac100680 0x7fbeac196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeac101870 0x7fbeac196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeaad9d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 0x7fbeac19dfb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbea402f540 tx=0x7fbea4009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbe9b7fe640 1 -- 172.21.15.112:0/3117764565 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea404a070 con 0x7fbeac102230 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/3117764565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac0688d0 con 0x7fbeac102230 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbe9b7fe640 1 -- 172.21.15.112:0/3117764565 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbea402fbd0 con 0x7fbeac102230 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbe9b7fe640 1 -- 172.21.15.112:0/3117764565 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea403a520 con 0x7fbeac102230 2024-04-24T23:15:26.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.534+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/3117764565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac068e70 con 0x7fbeac102230 2024-04-24T23:15:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.537+0000 7fbeb33fd740 1 Processor -- start 2024-04-24T23:15:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.537+0000 7fbe9b7fe640 1 -- 172.21.15.112:0/3117764565 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbea404d070 con 0x7fbeac102230 2024-04-24T23:15:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.537+0000 7fbeb33fd740 1 -- start start 2024-04-24T23:15:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d17640 0x5566b7d17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 0x5566b7d1cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 0x5566b7d1f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5566b7c77670 con 0x5566b7d17640 2024-04-24T23:15:26.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5566b7c1f970 con 0x5566b7d18030 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5566b7c73630 con 0x5566b7d1d1a0 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d17640 0x5566b7d17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d17640 0x5566b7d17a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52626/0 (socket says 172.21.15.112:52626) 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 -- 172.21.15.112:0/1657280252 learned_addr learned my addr 172.21.15.112:0/1657280252 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 0x5566b7d1cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 0x5566b7d1f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbe9b7fe640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe940a2b90 0x7fbe940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 -- 172.21.15.112:0/1657280252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 msgr2=0x5566b7d1f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 0x5566b7d1f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 -- 172.21.15.112:0/1657280252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 msgr2=0x5566b7d1cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 0x5566b7d1cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 -- 172.21.15.112:0/1657280252 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b7c43400 con 0x5566b7d17640 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbeaad9d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe940a2b90 0x7fbe940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d17640 0x5566b7d17a60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbea00123c0 tx=0x7fbea0009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb846c3b97d3d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbe9b7fe640 1 -- 172.21.15.112:0/3117764565 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbea40f1ba0 con 0x7fbeac102230 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.538+0000 7fbe98ff9640 1 -- 172.21.15.112:0/1657280252 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0004030 con 0x5566b7d17640 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbe98ff9640 1 -- 172.21.15.112:0/1657280252 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbea001b070 con 0x5566b7d17640 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbe98ff9640 1 -- 172.21.15.112:0/1657280252 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0003e60 con 0x5566b7d17640 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 -- 172.21.15.112:0/1657280252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d17640 msgr2=0x5566b7d17a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeaad9d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe940a2b90 0x7fbe940a5050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbea40ef300 tx=0x7fbea40e8ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d17640 0x5566b7d17a60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fbea00123c0 tx=0x7fbea0009740 comp rx=0 tx=0).stop 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 -- 172.21.15.112:0/1657280252 shutdown_connections 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 0x5566b7d1f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 0x5566b7d1cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/1657280252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d17640 0x5566b7d17a60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 -- 172.21.15.112:0/1657280252 >> 172.21.15.112:0/1657280252 conn(0x5566b7d0e8a0 msgr2=0x5566b7d0eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 -- 172.21.15.112:0/1657280252 shutdown_connections 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.539+0000 7fbeb33fd740 1 -- 172.21.15.112:0/1657280252 wait complete. 2024-04-24T23:15:26.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 1 -- start start 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 1 Processor -- start 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d21730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d18030 0x5566b7d21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d28cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5566b7d20150 con 0x5566b7d1d1a0 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5566b7c33050 con 0x5566b7d17640 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5566b7d202d0 con 0x5566b7d18030 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeb33fd740 10 client.0 _ll_register_callbacks cb 0x5566b798abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d28cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d28cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52630/0 (socket says 172.21.15.112:52630) 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 -- 172.21.15.112:0/577061546 learned_addr learned my addr 172.21.15.112:0/577061546 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d18030 0x5566b7d21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d21730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 -- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d18030 msgr2=0x5566b7d21c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d18030 0x5566b7d21c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 -- 172.21.15.112:0/577061546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 msgr2=0x5566b7d21730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d21730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.540+0000 7fbea9d9b640 1 -- 172.21.15.112:0/577061546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b7c04dc0 con 0x5566b7d1d1a0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d18030 0x5566b7d21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d28cb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbea00058f0 tx=0x7fbea00098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbbbe2f2099106c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbe91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbe91ffb640 1 -- 172.21.15.112:0/577061546 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0005030 con 0x5566b7d1d1a0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbe91ffb640 1 -- 172.21.15.112:0/577061546 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbea00051d0 con 0x5566b7d1d1a0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 1 -- 172.21.15.112:0/577061546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea001b070 con 0x5566b7d1d1a0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbe91ffb640 1 -- 172.21.15.112:0/577061546 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbea00053d0 con 0x5566b7d1d1a0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 10 client.15729 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 1 -- 172.21.15.112:0/577061546 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566b798b3f0 con 0x5566b7d1d1a0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbe91ffb640 1 -- 172.21.15.112:0/577061546 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbea001b070 con 0x5566b7d1d1a0 2024-04-24T23:15:26.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbe91ffb640 1 -- 172.21.15.112:0/577061546 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbea000f040 con 0x5566b7d1d1a0 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 10 client.15729 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 20 client.15729 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5566b7d2d320 0x5566b7d2f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 4 client.15729 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.541+0000 7fbeb33fd740 1 -- 172.21.15.112:0/577061546 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566b7c84bb0 con 0x5566b7d2d320 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.542+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5566b7d2d320 0x5566b7d2f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.542+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5566b7d2d320 0x5566b7d2f760 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9e5d27e572b322 server_cookie=f68ad3ce4e87b438 in_seq=0 out_seq=0 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.542+0000 7fbe91ffb640 10 client.15729 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbe91ffb640 1 -- 172.21.15.112:0/577061546 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566b7c84bb0 con 0x5566b7d2d320 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbe91ffb640 10 client.15729 handle_command_reply: tid=0 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 client.15729 shutdown 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 -- 172.21.15.112:0/577061546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 msgr2=0x5566b7d28cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d28cb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbea00058f0 tx=0x7fbea00098a0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 -- 172.21.15.112:0/577061546 shutdown_connections 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5566b7d2d320 0x5566b7d2f760 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d28cb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d18030 0x5566b7d21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbe91ffb640 0 client.15729 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/577061546 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d21730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 -- 172.21.15.112:0/577061546 >> 172.21.15.112:0/577061546 conn(0x5566b7d0e8a0 msgr2=0x5566b7d0db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 -- 172.21.15.112:0/577061546 shutdown_connections 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 1 -- 172.21.15.112:0/577061546 wait complete. 2024-04-24T23:15:26.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.543+0000 7fbeb33fd740 20 client.15729 trim_cache size 0 max 16384 2024-04-24T23:15:26.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.557+0000 7fbeb33fd740 1 Processor -- start 2024-04-24T23:15:26.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.557+0000 7fbeb33fd740 1 -- start start 2024-04-24T23:15:26.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.557+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d17640 0x5566b7d0d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.557+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 0x5566b7d28c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.557+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d0d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.557+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1a8400 con 0x5566b7d1d1a0 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.557+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1a9f40 con 0x5566b7d18030 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.557+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1a8ad0 con 0x5566b7d17640 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d0d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d0d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52632/0 (socket says 172.21.15.112:52632) 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 -- 172.21.15.112:0/123725470 learned_addr learned my addr 172.21.15.112:0/123725470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 0x5566b7d28c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d17640 0x5566b7d0d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 -- 172.21.15.112:0/123725470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d17640 msgr2=0x5566b7d0d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d17640 0x5566b7d0d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 -- 172.21.15.112:0/123725470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 msgr2=0x5566b7d28c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 0x5566b7d28c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 -- 172.21.15.112:0/123725470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b7a557d0 con 0x5566b7d1d1a0 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d0d7f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbea0001eb0 tx=0x7fbea0023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5dcd041106f2a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbe93fff640 1 -- 172.21.15.112:0/123725470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0003bd0 con 0x5566b7d1d1a0 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbe93fff640 1 -- 172.21.15.112:0/123725470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbea001b040 con 0x5566b7d1d1a0 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.558+0000 7fbe93fff640 1 -- 172.21.15.112:0/123725470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0021670 con 0x5566b7d1d1a0 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- 172.21.15.112:0/123725470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 msgr2=0x5566b7d0d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d0d7f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fbea0001eb0 tx=0x7fbea0023040 comp rx=0 tx=0).stop 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- 172.21.15.112:0/123725470 shutdown_connections 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d1d1a0 0x5566b7d0d7f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d18030 0x5566b7d28c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/123725470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d17640 0x5566b7d0d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- 172.21.15.112:0/123725470 >> 172.21.15.112:0/123725470 conn(0x5566b7d0e8a0 msgr2=0x5566b7d19360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- 172.21.15.112:0/123725470 shutdown_connections 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- 172.21.15.112:0/123725470 wait complete. 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- start start 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 Processor -- start 2024-04-24T23:15:26.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d25ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d18030 0x5566b7d26420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 0x5566b7d27430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1a9870 con 0x5566b7d18030 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1a91a0 con 0x5566b7d17640 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1a63e0 con 0x5566b7d1d1a0 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.559+0000 7fbeb33fd740 10 client.0 _ll_register_callbacks cb 0x7fbea0009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d18030 0x5566b7d26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d18030 0x5566b7d26420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52636/0 (socket says 172.21.15.112:52636) 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 -- 172.21.15.112:0/620627261 learned_addr learned my addr 172.21.15.112:0/620627261 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d25ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 -- 172.21.15.112:0/620627261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 msgr2=0x5566b7d27430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 0x5566b7d27430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 -- 172.21.15.112:0/620627261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 msgr2=0x5566b7d25ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d25ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 -- 172.21.15.112:0/620627261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b7d27a20 con 0x5566b7d18030 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d25ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeaa59c640 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d18030 0x5566b7d26420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0058c0 tx=0x7fbe9c005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0848b7a99b36749 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbe91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbe91ffb640 1 -- 172.21.15.112:0/620627261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c004e00 con 0x5566b7d18030 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbe91ffb640 1 -- 172.21.15.112:0/620627261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbe9c01b290 con 0x5566b7d18030 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeb33fd740 1 -- 172.21.15.112:0/620627261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea001b040 con 0x5566b7d18030 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbe91ffb640 1 -- 172.21.15.112:0/620627261 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe9c02d050 con 0x5566b7d18030 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeb33fd740 10 client.15735 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbeb33fd740 1 -- 172.21.15.112:0/620627261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566b7d28e20 con 0x5566b7d18030 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.560+0000 7fbe91ffb640 1 -- 172.21.15.112:0/620627261 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbea001b040 con 0x5566b7d18030 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbe91ffb640 1 -- 172.21.15.112:0/620627261 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe9c024b70 con 0x5566b7d18030 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbeb33fd740 10 client.15735 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbeb33fd740 10 client.15735 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbeb33fd740 20 client.15735 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5566b7d93ea0 0x5566b7d962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbeb33fd740 4 client.15735 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbeb33fd740 1 -- 172.21.15.112:0/620627261 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566b7c7ef00 con 0x5566b7d93ea0 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5566b7d93ea0 0x5566b7d962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbea9d9b640 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5566b7d93ea0 0x5566b7d962e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dddbb34e0e467441 server_cookie=28d5e4fac2010d18 in_seq=0 out_seq=0 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbe91ffb640 10 client.15735 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbe91ffb640 1 -- 172.21.15.112:0/620627261 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566b7c7ef00 con 0x5566b7d93ea0 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.561+0000 7fbe91ffb640 10 client.15735 handle_command_reply: tid=0 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 client.15735 shutdown 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 -- 172.21.15.112:0/620627261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d18030 msgr2=0x5566b7d26420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d18030 0x5566b7d26420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0058c0 tx=0x7fbe9c005410 comp rx=0 tx=0).stop 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 -- 172.21.15.112:0/620627261 shutdown_connections 2024-04-24T23:15:26.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5566b7d93ea0 0x5566b7d962e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7d1d1a0 0x5566b7d27430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbe91ffb640 0 client.15735 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5566b7d18030 0x5566b7d26420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 --2- 172.21.15.112:0/620627261 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5566b7d17640 0x5566b7d25ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 -- 172.21.15.112:0/620627261 >> 172.21.15.112:0/620627261 conn(0x5566b7d0e8a0 msgr2=0x5566b7d117b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 -- 172.21.15.112:0/620627261 shutdown_connections 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 1 -- 172.21.15.112:0/620627261 wait complete. 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb33fd740 20 client.15735 trim_cache size 0 max 16384 2024-04-24T23:15:26.570 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:26.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe940a2b90 msgr2=0x7fbe940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe940a2b90 0x7fbe940a5050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbea40ef300 tx=0x7fbea40e8ba0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 msgr2=0x7fbeac19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 0x7fbeac19dfb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbea402f540 tx=0x7fbea4009890 comp rx=0 tx=0).stop 2024-04-24T23:15:26.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/3117764565 shutdown_connections 2024-04-24T23:15:26.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe940a2b90 0x7fbe940a5050 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbeac102230 0x7fbeac19dfb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbeac101870 0x7fbeac196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 --2- 172.21.15.112:0/3117764565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbeac100680 0x7fbeac196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/3117764565 >> 172.21.15.112:0/3117764565 conn(0x7fbeac0fbbf0 msgr2=0x7fbeac104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/3117764565 shutdown_connections 2024-04-24T23:15:26.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.562+0000 7fbeb0a4d640 1 -- 172.21.15.112:0/3117764565 wait complete. 2024-04-24T23:15:26.575 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T23:15:26.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 -- 172.21.15.112:0/3611522030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 msgr2=0x7fbe48100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/3611522030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 0x7fbe48100aa0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009900 tx=0x7fbe4402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 -- 172.21.15.112:0/3611522030 shutdown_connections 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/3611522030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe48102230 0x7fbe48108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/3611522030 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe48101870 0x7fbe48101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/3611522030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 0x7fbe48100aa0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 -- 172.21.15.112:0/3611522030 >> 172.21.15.112:0/3611522030 conn(0x7fbe480fbbf0 msgr2=0x7fbe480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 -- 172.21.15.112:0/3611522030 shutdown_connections 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.676+0000 7fbe4f42a640 1 -- 172.21.15.112:0/3611522030 wait complete. 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4f42a640 1 Processor -- start 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4f42a640 1 -- start start 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4f42a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 0x7fbe48104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4f42a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe48101870 0x7fbe48102c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4f42a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe48102230 0x7fbe48103190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4f42a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810c420 con 0x7fbe48100680 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4f42a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810c2a0 con 0x7fbe48101870 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4f42a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4810c5a0 con 0x7fbe48102230 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d1c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 0x7fbe48104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d1c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 0x7fbe48104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52656/0 (socket says 172.21.15.112:52656) 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/963397578 learned_addr learned my addr 172.21.15.112:0/963397578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe48101870 0x7fbe48102c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d9c7640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe48102230 0x7fbe48103190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/963397578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe48102230 msgr2=0x7fbe48103190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe48102230 0x7fbe48103190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/963397578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe48101870 msgr2=0x7fbe48102c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe48101870 0x7fbe48102c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/963397578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe48103a20 con 0x7fbe48100680 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.677+0000 7fbe4d9c7640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe48102230 0x7fbe48103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:26.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.678+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 0x7fbe48104610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009230 tx=0x7fbe440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.678+0000 7fbe3e7fc640 1 -- 172.21.15.112:0/963397578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4403f030 con 0x7fbe48100680 2024-04-24T23:15:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.678+0000 7fbe4f42a640 1 -- 172.21.15.112:0/963397578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe481a1c10 con 0x7fbe48100680 2024-04-24T23:15:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.678+0000 7fbe3e7fc640 1 -- 172.21.15.112:0/963397578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe44003d00 con 0x7fbe48100680 2024-04-24T23:15:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.678+0000 7fbe3e7fc640 1 -- 172.21.15.112:0/963397578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44035070 con 0x7fbe48100680 2024-04-24T23:15:26.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.678+0000 7fbe4f42a640 1 -- 172.21.15.112:0/963397578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe481a2160 con 0x7fbe48100680 2024-04-24T23:15:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.680+0000 7fbe3e7fc640 1 -- 172.21.15.112:0/963397578 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbe44042030 con 0x7fbe48100680 2024-04-24T23:15:26.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe52215740 1 Processor -- start 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe52215740 1 -- start start 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe52215740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 0x5587d9b7e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe52215740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe52215740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b86220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe52215740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587d9ae2670 con 0x5587d9b7ec20 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe52215740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587d9a8a970 con 0x5587d9b83d90 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe52215740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587d9ade630 con 0x5587d9b7e230 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d1c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe3e7fc640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe340a2b90 0x7fbe340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d1c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b83720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52670/0 (socket says 172.21.15.112:52670) 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/2884979719 learned_addr learned my addr 172.21.15.112:0/2884979719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d9c7640 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 0x5587d9b7e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d9c7640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe340a2b90 0x7fbe340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b86220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe3e7fc640 1 -- 172.21.15.112:0/963397578 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbe440ef9e0 con 0x7fbe48100680 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/2884979719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 msgr2=0x5587d9b7e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 0x5587d9b7e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/2884979719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 msgr2=0x5587d9b86220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b86220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/2884979719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d9aae400 con 0x5587d9b7ec20 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.681+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b86220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.682+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b83720 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbe440bc4b0 tx=0x7fbe4402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3976ce266802661 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.682+0000 7fbe4d9c7640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe340a2b90 0x7fbe340a5050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbe40013bd0 tx=0x7fbe40007980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.682+0000 7fbe337fe640 1 -- 172.21.15.112:0/2884979719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44049070 con 0x5587d9b7ec20 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.682+0000 7fbe337fe640 1 -- 172.21.15.112:0/2884979719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe440be070 con 0x5587d9b7ec20 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.682+0000 7fbe337fe640 1 -- 172.21.15.112:0/2884979719 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4403b460 con 0x5587d9b7ec20 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.682+0000 7fbe52215740 1 -- 172.21.15.112:0/2884979719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 msgr2=0x5587d9b83720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.682+0000 7fbe52215740 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b83720 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbe440bc4b0 tx=0x7fbe4402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 -- 172.21.15.112:0/2884979719 shutdown_connections 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b86220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b83720 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 --2- 172.21.15.112:0/2884979719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 0x5587d9b7e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 -- 172.21.15.112:0/2884979719 >> 172.21.15.112:0/2884979719 conn(0x5587d9b798a0 msgr2=0x5587d9b79ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 -- 172.21.15.112:0/2884979719 shutdown_connections 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 -- 172.21.15.112:0/2884979719 wait complete. 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 -- start start 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 Processor -- start 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.683+0000 7fbe52215740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b87fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7ec20 0x5587d9b88500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b8f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587d9ae2670 con 0x5587d9b7e230 2024-04-24T23:15:26.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587d9a9e050 con 0x5587d9b83d90 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587d9b86b00 con 0x5587d9b7ec20 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 10 client.0 _ll_register_callbacks cb 0x5587d97f5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b87fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b87fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52682/0 (socket says 172.21.15.112:52682) 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/3842863756 learned_addr learned my addr 172.21.15.112:0/3842863756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d9c7640 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b8f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7ec20 0x5587d9b88500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/3842863756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7ec20 msgr2=0x5587d9b88500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7ec20 0x5587d9b88500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/3842863756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 msgr2=0x5587d9b8f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b8f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/3842863756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d9a6fdc0 con 0x5587d9b7e230 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b87fc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbe440b82b0 tx=0x7fbe440045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc31af0fd31da0a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe30ff9640 1 -- 172.21.15.112:0/3842863756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44049070 con 0x5587d9b7e230 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe30ff9640 1 -- 172.21.15.112:0/3842863756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbe4403b460 con 0x5587d9b7e230 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 1 -- 172.21.15.112:0/3842863756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe440be070 con 0x5587d9b7e230 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe30ff9640 1 -- 172.21.15.112:0/3842863756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe440c0200 con 0x5587d9b7e230 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 10 client.15747 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.684+0000 7fbe52215740 1 -- 172.21.15.112:0/3842863756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587d97f63f0 con 0x5587d9b7e230 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe30ff9640 1 -- 172.21.15.112:0/3842863756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe440be070 con 0x5587d9b7e230 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe30ff9640 1 -- 172.21.15.112:0/3842863756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe440c4630 con 0x5587d9b7e230 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe52215740 10 client.15747 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe52215740 10 client.15747 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe52215740 20 client.15747 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe52215740 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5587d9b93bd0 0x5587d9b96000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe52215740 4 client.15747 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:26.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe52215740 1 -- 172.21.15.112:0/3842863756 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587d9aefbb0 con 0x5587d9b93bd0 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.685+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5587d9b93bd0 0x5587d9b96000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5587d9b93bd0 0x5587d9b96000 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1340e15fc425c4 server_cookie=6b65a74f9001c1e3 in_seq=0 out_seq=0 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe30ff9640 10 client.15747 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe30ff9640 1 -- 172.21.15.112:0/3842863756 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587d9aefbb0 con 0x5587d9b93bd0 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe30ff9640 10 client.15747 handle_command_reply: tid=0 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 client.15747 shutdown 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 -- 172.21.15.112:0/3842863756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 msgr2=0x5587d9b87fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b87fc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbe440b82b0 tx=0x7fbe440045e0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 -- 172.21.15.112:0/3842863756 shutdown_connections 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5587d9b93bd0 0x5587d9b96000 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b8f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe30ff9640 0 client.15747 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7ec20 0x5587d9b88500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 --2- 172.21.15.112:0/3842863756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b87fc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.686+0000 7fbe52215740 1 -- 172.21.15.112:0/3842863756 >> 172.21.15.112:0/3842863756 conn(0x5587d9b798a0 msgr2=0x5587d9b78b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.687+0000 7fbe52215740 1 -- 172.21.15.112:0/3842863756 shutdown_connections 2024-04-24T23:15:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.687+0000 7fbe52215740 1 -- 172.21.15.112:0/3842863756 wait complete. 2024-04-24T23:15:26.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.687+0000 7fbe52215740 20 client.15747 trim_cache size 0 max 16384 2024-04-24T23:15:26.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe52215740 1 Processor -- start 2024-04-24T23:15:26.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe52215740 1 -- start start 2024-04-24T23:15:26.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe52215740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe52215740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b7ec20 0x5587d9b8f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe52215740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b83d90 0x5587d9b787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe52215740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481ac760 con 0x5587d9b7e230 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe52215740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481ae2a0 con 0x5587d9b7ec20 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe52215740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481ace30 con 0x5587d9b83d90 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe4d1c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe4d1c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b782b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52686/0 (socket says 172.21.15.112:52686) 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.700+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/937874154 learned_addr learned my addr 172.21.15.112:0/937874154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/937874154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b83d90 msgr2=0x5587d9b787f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/937874154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b83d90 0x5587d9b787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/937874154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b7ec20 msgr2=0x5587d9b8f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/937874154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b7ec20 0x5587d9b8f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe4d1c6640 1 -- 172.21.15.112:0/937874154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d98c07d0 con 0x5587d9b7e230 2024-04-24T23:15:26.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/937874154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b782b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe440bbde0 tx=0x7fbe44009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bb9206c1f23675 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe32ffd640 1 -- 172.21.15.112:0/937874154 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44049070 con 0x5587d9b7e230 2024-04-24T23:15:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe32ffd640 1 -- 172.21.15.112:0/937874154 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe440be050 con 0x5587d9b7e230 2024-04-24T23:15:26.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.701+0000 7fbe32ffd640 1 -- 172.21.15.112:0/937874154 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe440bc550 con 0x5587d9b7e230 2024-04-24T23:15:26.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 -- 172.21.15.112:0/937874154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 msgr2=0x5587d9b782b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 --2- 172.21.15.112:0/937874154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b782b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbe440bbde0 tx=0x7fbe44009260 comp rx=0 tx=0).stop 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 -- 172.21.15.112:0/937874154 shutdown_connections 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 --2- 172.21.15.112:0/937874154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b83d90 0x5587d9b787f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 --2- 172.21.15.112:0/937874154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b7ec20 0x5587d9b8f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 --2- 172.21.15.112:0/937874154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7e230 0x5587d9b782b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 -- 172.21.15.112:0/937874154 >> 172.21.15.112:0/937874154 conn(0x5587d9b798a0 msgr2=0x5587d9b7d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 -- 172.21.15.112:0/937874154 shutdown_connections 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 -- 172.21.15.112:0/937874154 wait complete. 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 -- start start 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.702+0000 7fbe52215740 1 Processor -- start 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe52215740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 0x5587d9b8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe52215740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b8cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe52215740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b8de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe52215740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481adbd0 con 0x5587d9b7ec20 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe52215740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481ad500 con 0x5587d9b83d90 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe52215740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481aa740 con 0x5587d9b7e230 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe52215740 10 client.0 _ll_register_callbacks cb 0x7fbe40007e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b8cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b8cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52702/0 (socket says 172.21.15.112:52702) 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 -- 172.21.15.112:0/1193912624 learned_addr learned my addr 172.21.15.112:0/1193912624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 0x5587d9b8c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 -- 172.21.15.112:0/1193912624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 msgr2=0x5587d9b8c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 0x5587d9b8c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 -- 172.21.15.112:0/1193912624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 msgr2=0x5587d9b8de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b8de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 -- 172.21.15.112:0/1193912624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d9b8e430 con 0x5587d9b7ec20 2024-04-24T23:15:26.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe4c9c5640 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b8cea0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5587d9b96800 tx=0x7fbe38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a397ee6008e8bcbf server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.703+0000 7fbe30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe30ff9640 1 -- 172.21.15.112:0/1193912624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38004440 con 0x5587d9b7ec20 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe30ff9640 1 -- 172.21.15.112:0/1193912624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbe3801f110 con 0x5587d9b7ec20 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 1 -- 172.21.15.112:0/1193912624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe440be050 con 0x5587d9b7ec20 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe30ff9640 1 -- 172.21.15.112:0/1193912624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe3802e050 con 0x5587d9b7ec20 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 10 client.15753 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 1 -- 172.21.15.112:0/1193912624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5587d9b8f830 con 0x5587d9b7ec20 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe30ff9640 1 -- 172.21.15.112:0/1193912624 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe440be050 con 0x5587d9b7ec20 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe30ff9640 1 -- 172.21.15.112:0/1193912624 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe38003a70 con 0x5587d9b7ec20 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 10 client.15753 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 10 client.15753 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 20 client.15753 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5587d9bfa930 0x5587d9bfcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 4 client.15753 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe52215740 1 -- 172.21.15.112:0/1193912624 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587d9ae9f00 con 0x5587d9bfa930 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.704+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5587d9bfa930 0x5587d9bfcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe30ff9640 10 client.15753 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe4d1c6640 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5587d9bfa930 0x5587d9bfcd50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=13fcaf3714b4210d server_cookie=c25a4655e0196d0f in_seq=0 out_seq=0 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe30ff9640 1 -- 172.21.15.112:0/1193912624 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587d9ae9f00 con 0x5587d9bfa930 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe30ff9640 10 client.15753 handle_command_reply: tid=0 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 client.15753 shutdown 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 -- 172.21.15.112:0/1193912624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 msgr2=0x5587d9b8cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b8cea0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5587d9b96800 tx=0x7fbe38011810 comp rx=0 tx=0).stop 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 -- 172.21.15.112:0/1193912624 shutdown_connections 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5587d9bfa930 0x5587d9bfcd50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587d9b83d90 0x5587d9b8de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe30ff9640 0 client.15753 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:15:26.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587d9b7ec20 0x5587d9b8cea0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 --2- 172.21.15.112:0/1193912624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587d9b7e230 0x5587d9b8c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 -- 172.21.15.112:0/1193912624 >> 172.21.15.112:0/1193912624 conn(0x5587d9b798a0 msgr2=0x5587d9b7f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 -- 172.21.15.112:0/1193912624 shutdown_connections 2024-04-24T23:15:26.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 1 -- 172.21.15.112:0/1193912624 wait complete. 2024-04-24T23:15:26.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.705+0000 7fbe52215740 20 client.15753 trim_cache size 0 max 16384 2024-04-24T23:15:26.714 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 -- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe340a2b90 msgr2=0x7fbe340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe340a2b90 0x7fbe340a5050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbe40013bd0 tx=0x7fbe40007980 comp rx=0 tx=0).stop 2024-04-24T23:15:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 -- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 msgr2=0x7fbe48104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 0x7fbe48104610 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009230 tx=0x7fbe440042f0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 -- 172.21.15.112:0/963397578 shutdown_connections 2024-04-24T23:15:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe340a2b90 0x7fbe340a5050 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe48102230 0x7fbe48103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe48101870 0x7fbe48102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 --2- 172.21.15.112:0/963397578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe48100680 0x7fbe48104610 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 -- 172.21.15.112:0/963397578 >> 172.21.15.112:0/963397578 conn(0x7fbe480fbbf0 msgr2=0x7fbe480fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 -- 172.21.15.112:0/963397578 shutdown_connections 2024-04-24T23:15:26.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.706+0000 7fbe4f42a640 1 -- 172.21.15.112:0/963397578 wait complete. 2024-04-24T23:15:26.719 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.822+0000 7f1611929640 1 -- 172.21.15.112:0/2522124536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c102230 msgr2=0x7f160c108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.822+0000 7f1611929640 1 --2- 172.21.15.112:0/2522124536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c102230 0x7f160c108ac0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1604009ca0 tx=0x7f160402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 -- 172.21.15.112:0/2522124536 shutdown_connections 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 --2- 172.21.15.112:0/2522124536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c102230 0x7f160c108ac0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 --2- 172.21.15.112:0/2522124536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f160c101870 0x7f160c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 --2- 172.21.15.112:0/2522124536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f160c100680 0x7f160c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 -- 172.21.15.112:0/2522124536 >> 172.21.15.112:0/2522124536 conn(0x7f160c0fbbf0 msgr2=0x7f160c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 -- 172.21.15.112:0/2522124536 shutdown_connections 2024-04-24T23:15:26.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 -- 172.21.15.112:0/2522124536 wait complete. 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 Processor -- start 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.823+0000 7f1611929640 1 -- start start 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1611929640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c100680 0x7f160c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1611929640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f160c101870 0x7f160c19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1611929640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f160c102230 0x7f160c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1611929640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f160c10c430 con 0x7f160c100680 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1611929640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f160c10c2b0 con 0x7f160c102230 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1611929640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f160c10c5b0 con 0x7f160c101870 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c100680 0x7f160c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c100680 0x7f160c069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52724/0 (socket says 172.21.15.112:52724) 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 -- 172.21.15.112:0/2424655156 learned_addr learned my addr 172.21.15.112:0/2424655156 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 -- 172.21.15.112:0/2424655156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f160c101870 msgr2=0x7f160c19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1603fff640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f160c101870 0x7f160c19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f160c101870 0x7f160c19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 -- 172.21.15.112:0/2424655156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f160c102230 msgr2=0x7f160c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f160c102230 0x7f160c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 -- 172.21.15.112:0/2424655156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c19f790 con 0x7f160c100680 2024-04-24T23:15:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f160affd640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c100680 0x7f160c069440 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00c970 tx=0x7f15fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1610927640 1 -- 172.21.15.112:0/2424655156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc004220 con 0x7f160c100680 2024-04-24T23:15:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1610927640 1 -- 172.21.15.112:0/2424655156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15fc0043c0 con 0x7f160c100680 2024-04-24T23:15:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1611929640 1 -- 172.21.15.112:0/2424655156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c19fa20 con 0x7f160c100680 2024-04-24T23:15:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1610927640 1 -- 172.21.15.112:0/2424655156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc00f6b0 con 0x7f160c100680 2024-04-24T23:15:26.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.824+0000 7f1611929640 1 -- 172.21.15.112:0/2424655156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c19ff70 con 0x7f160c100680 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1610927640 1 -- 172.21.15.112:0/2424655156 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f15fc0037a0 con 0x7f160c100680 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1614636740 1 Processor -- start 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1614636740 1 -- start start 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1614636740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de34e640 0x5635de34ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1614636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de353b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1614636740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de3541a0 0x5635de356630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1614636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5635de2ae670 con 0x5635de34f030 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1614636740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5635de256970 con 0x5635de34e640 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f1614636740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5635de2aa630 con 0x5635de3541a0 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f160affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de353b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f160affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de353b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52740/0 (socket says 172.21.15.112:52740) 2024-04-24T23:15:26.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f160affd640 1 -- 172.21.15.112:0/3227462596 learned_addr learned my addr 172.21.15.112:0/3227462596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.827+0000 7f160b7fe640 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de34e640 0x5635de34ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f160affd640 1 -- 172.21.15.112:0/3227462596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de3541a0 msgr2=0x5635de356630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f1603fff640 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de3541a0 0x5635de356630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f160affd640 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de3541a0 0x5635de356630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f160affd640 1 -- 172.21.15.112:0/3227462596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de34e640 msgr2=0x5635de34ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f160affd640 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de34e640 0x5635de34ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f160affd640 1 -- 172.21.15.112:0/3227462596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635de27a400 con 0x5635de34f030 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f160b7fe640 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de34e640 0x5635de34ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f1610927640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15f80a2b90 0x7f15f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f160affd640 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de353b30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0d7500 tx=0x7f15fc0d79c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4febb23ab656ccfc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f160b7fe640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15f80a2b90 0x7f15f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f1610927640 1 -- 172.21.15.112:0/2424655156 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f15fc0cf220 con 0x7f160c100680 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f1601ffb640 1 -- 172.21.15.112:0/3227462596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc097930 con 0x5635de34f030 2024-04-24T23:15:26.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f1601ffb640 1 -- 172.21.15.112:0/3227462596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15fc09bd90 con 0x5635de34f030 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.828+0000 7f1601ffb640 1 -- 172.21.15.112:0/3227462596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc09aff0 con 0x5635de34f030 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 -- 172.21.15.112:0/3227462596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 msgr2=0x5635de353b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f160b7fe640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15f80a2b90 0x7f15f80a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5635de34f890 tx=0x7f1604005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de353b30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0d7500 tx=0x7f15fc0d79c0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 -- 172.21.15.112:0/3227462596 shutdown_connections 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de3541a0 0x5635de356630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de353b30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 --2- 172.21.15.112:0/3227462596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de34e640 0x5635de34ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 -- 172.21.15.112:0/3227462596 >> 172.21.15.112:0/3227462596 conn(0x5635de3458a0 msgr2=0x5635de345ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 -- 172.21.15.112:0/3227462596 shutdown_connections 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 -- 172.21.15.112:0/3227462596 wait complete. 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 -- start start 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.829+0000 7f1614636740 1 Processor -- start 2024-04-24T23:15:26.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1614636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34e640 0x5635de3587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1614636740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34f030 0x5635de358ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1614636740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de35fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1614636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5635de357150 con 0x5635de34e640 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1614636740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5635de26a050 con 0x5635de3541a0 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1614636740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5635de3572d0 con 0x5635de34f030 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1614636740 10 client.0 _ll_register_callbacks cb 0x5635ddfc1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34e640 0x5635de3587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34e640 0x5635de3587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52750/0 (socket says 172.21.15.112:52750) 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 -- 172.21.15.112:0/3030829174 learned_addr learned my addr 172.21.15.112:0/3030829174 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1603fff640 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34f030 0x5635de358ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 -- 172.21.15.112:0/3030829174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34f030 msgr2=0x5635de358ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34f030 0x5635de358ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 -- 172.21.15.112:0/3030829174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 msgr2=0x5635de35fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160b7fe640 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de35fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de35fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 -- 172.21.15.112:0/3030829174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635de23bdc0 con 0x5635de34e640 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1603fff640 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34f030 0x5635de358ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f160affd640 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34e640 0x5635de3587a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0cc980 tx=0x7f15fc003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7ed62f98458f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f15eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f15eaffd640 1 -- 172.21.15.112:0/3030829174 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc01d040 con 0x5635de34e640 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f15eaffd640 1 -- 172.21.15.112:0/3030829174 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f15fc099990 con 0x5635de34e640 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.830+0000 7f1614636740 1 -- 172.21.15.112:0/3030829174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc09bd90 con 0x5635de34e640 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f15eaffd640 1 -- 172.21.15.112:0/3030829174 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15fc09b1d0 con 0x5635de34e640 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1614636740 10 client.15765 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1614636740 1 -- 172.21.15.112:0/3030829174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635ddfc23f0 con 0x5635de34e640 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f15eaffd640 1 -- 172.21.15.112:0/3030829174 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15fc09bd90 con 0x5635de34e640 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f15eaffd640 1 -- 172.21.15.112:0/3030829174 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f15fc0a3e70 con 0x5635de34e640 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1614636740 10 client.15765 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1614636740 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1614636740 20 client.15765 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1614636740 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5635de364210 0x5635de366600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1614636740 4 client.15765 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1614636740 1 -- 172.21.15.112:0/3030829174 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635de2bbbb0 con 0x5635de364210 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.831+0000 7f1603fff640 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5635de364210 0x5635de366600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.832+0000 7f15eaffd640 10 client.15765 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.832+0000 7f1603fff640 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5635de364210 0x5635de366600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb5be1890cfc1028 server_cookie=df6423c0d142951 in_seq=0 out_seq=0 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.832+0000 7f15eaffd640 1 -- 172.21.15.112:0/3030829174 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635de2bbbb0 con 0x5635de364210 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.832+0000 7f15eaffd640 10 client.15765 handle_command_reply: tid=0 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 client.15765 shutdown 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 -- 172.21.15.112:0/3030829174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34e640 msgr2=0x5635de3587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34e640 0x5635de3587a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0cc980 tx=0x7f15fc003b40 comp rx=0 tx=0).stop 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 -- 172.21.15.112:0/3030829174 shutdown_connections 2024-04-24T23:15:26.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5635de364210 0x5635de366600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de35fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34f030 0x5635de358ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f15eaffd640 0 client.15765 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:15:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 --2- 172.21.15.112:0/3030829174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34e640 0x5635de3587a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 -- 172.21.15.112:0/3030829174 >> 172.21.15.112:0/3030829174 conn(0x5635de3458a0 msgr2=0x5635de344b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 -- 172.21.15.112:0/3030829174 shutdown_connections 2024-04-24T23:15:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 1 -- 172.21.15.112:0/3030829174 wait complete. 2024-04-24T23:15:26.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.833+0000 7f1614636740 20 client.15765 trim_cache size 0 max 16384 2024-04-24T23:15:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.846+0000 7f1614636740 1 Processor -- start 2024-04-24T23:15:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.846+0000 7f1614636740 1 -- start start 2024-04-24T23:15:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.846+0000 7f1614636740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 0x5635de34faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.846+0000 7f1614636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.846+0000 7f1614636740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de34ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.846+0000 7f1614636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1a8480 con 0x5635de34f030 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.846+0000 7f1614636740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1a9fc0 con 0x5635de3541a0 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.846+0000 7f1614636740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1a8b50 con 0x5635de34e640 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52752/0 (socket says 172.21.15.112:52752) 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 -- 172.21.15.112:0/2081164610 learned_addr learned my addr 172.21.15.112:0/2081164610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f160affd640 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 0x5635de34faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f160b7fe640 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de34ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 -- 172.21.15.112:0/2081164610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 msgr2=0x5635de34faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 0x5635de34faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 -- 172.21.15.112:0/2081164610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 msgr2=0x5635de34ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de34ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 -- 172.21.15.112:0/2081164610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635de08c7d0 con 0x5635de34f030 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f1603fff640 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35fcb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f160c0691b0 tx=0x7f15f4005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43572e2e93d6709d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f16017fa640 1 -- 172.21.15.112:0/2081164610 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f401f030 con 0x5635de34f030 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f16017fa640 1 -- 172.21.15.112:0/2081164610 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15f401b8a0 con 0x5635de34f030 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.847+0000 7f16017fa640 1 -- 172.21.15.112:0/2081164610 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4004b00 con 0x5635de34f030 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 -- 172.21.15.112:0/2081164610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 msgr2=0x5635de35fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35fcb0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f160c0691b0 tx=0x7f15f4005f50 comp rx=0 tx=0).stop 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 -- 172.21.15.112:0/2081164610 shutdown_connections 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de34ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35fcb0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 --2- 172.21.15.112:0/2081164610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 0x5635de34faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 -- 172.21.15.112:0/2081164610 >> 172.21.15.112:0/2081164610 conn(0x5635de3458a0 msgr2=0x5635de344b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 -- 172.21.15.112:0/2081164610 shutdown_connections 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 -- 172.21.15.112:0/2081164610 wait complete. 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.848+0000 7f1614636740 1 -- start start 2024-04-24T23:15:26.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1614636740 1 Processor -- start 2024-04-24T23:15:26.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1614636740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 0x5635de35d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1614636740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1614636740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de35ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1614636740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1a98f0 con 0x5635de34f030 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1614636740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1a9220 con 0x5635de3541a0 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1614636740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1a6460 con 0x5635de34e640 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1614636740 10 client.0 _ll_register_callbacks cb 0x7f15fc09f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1603fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1603fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52758/0 (socket says 172.21.15.112:52758) 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1603fff640 1 -- 172.21.15.112:0/2938384975 learned_addr learned my addr 172.21.15.112:0/2938384975 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f160b7fe640 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de35ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f160affd640 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 0x5635de35d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1603fff640 1 -- 172.21.15.112:0/2938384975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 msgr2=0x5635de35d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1603fff640 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 0x5635de35d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1603fff640 1 -- 172.21.15.112:0/2938384975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 msgr2=0x5635de35ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1603fff640 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de35ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.849+0000 7f1603fff640 1 -- 172.21.15.112:0/2938384975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635de35f2d0 con 0x5635de34f030 2024-04-24T23:15:26.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1603fff640 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35dde0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f15f4016010 tx=0x7f15f4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b53102a474a8498 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f15eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f15eaffd640 1 -- 172.21.15.112:0/2938384975 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f401b5a0 con 0x5635de34f030 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f15eaffd640 1 -- 172.21.15.112:0/2938384975 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f15f4004880 con 0x5635de34f030 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 1 -- 172.21.15.112:0/2938384975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15f401b8a0 con 0x5635de34f030 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f15eaffd640 1 -- 172.21.15.112:0/2938384975 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15f4036050 con 0x5635de34f030 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 1 -- 172.21.15.112:0/2938384975 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635de3ca970 con 0x5635de34f030 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f15eaffd640 1 -- 172.21.15.112:0/2938384975 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15f401b8a0 con 0x5635de34f030 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f15eaffd640 1 -- 172.21.15.112:0/2938384975 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f15f4024440 con 0x5635de34f030 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 10 client.15771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 20 client.15771 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5635de3cb2e0 0x5635de3cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 4 client.15771 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.850+0000 7f1614636740 1 -- 172.21.15.112:0/2938384975 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635de2b5f00 con 0x5635de3cb2e0 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f160affd640 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5635de3cb2e0 0x5635de3cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f15eaffd640 10 client.15771 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f160affd640 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5635de3cb2e0 0x5635de3cd700 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55d670a10ba0502f server_cookie=5e0b05e05332fb75 in_seq=0 out_seq=0 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f15eaffd640 1 -- 172.21.15.112:0/2938384975 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635de2b5f00 con 0x5635de3cb2e0 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f15eaffd640 10 client.15771 handle_command_reply: tid=0 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 client.15771 shutdown 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 -- 172.21.15.112:0/2938384975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 msgr2=0x5635de35dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35dde0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f15f4016010 tx=0x7f15f4005b00 comp rx=0 tx=0).stop 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 -- 172.21.15.112:0/2938384975 shutdown_connections 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5635de3cb2e0 0x5635de3cd700 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5635de3541a0 0x5635de35ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5635de34f030 0x5635de35dde0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 --2- 172.21.15.112:0/2938384975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635de34e640 0x5635de35d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f15eaffd640 0 client.15771 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:15:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.851+0000 7f1614636740 1 -- 172.21.15.112:0/2938384975 >> 172.21.15.112:0/2938384975 conn(0x5635de3458a0 msgr2=0x5635de3442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1614636740 1 -- 172.21.15.112:0/2938384975 shutdown_connections 2024-04-24T23:15:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1614636740 1 -- 172.21.15.112:0/2938384975 wait complete. 2024-04-24T23:15:26.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1614636740 20 client.15771 trim_cache size 0 max 16384 2024-04-24T23:15:26.860 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 -- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15f80a2b90 msgr2=0x7f15f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15f80a2b90 0x7f15f80a5050 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5635de34f890 tx=0x7f1604005f80 comp rx=0 tx=0).stop 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 -- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c100680 msgr2=0x7f160c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c100680 0x7f160c069440 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00c970 tx=0x7f15fc00ce30 comp rx=0 tx=0).stop 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 -- 172.21.15.112:0/2424655156 shutdown_connections 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15f80a2b90 0x7f15f80a5050 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f160c102230 0x7f160c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f160c101870 0x7f160c19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 --2- 172.21.15.112:0/2424655156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f160c100680 0x7f160c069440 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 -- 172.21.15.112:0/2424655156 >> 172.21.15.112:0/2424655156 conn(0x7f160c0fbbf0 msgr2=0x7f160c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 -- 172.21.15.112:0/2424655156 shutdown_connections 2024-04-24T23:15:26.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.852+0000 7f1611929640 1 -- 172.21.15.112:0/2424655156 wait complete. 2024-04-24T23:15:26.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.969+0000 7f75f93b7640 1 -- 172.21.15.112:0/505184223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4101870 msgr2=0x7f75f4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.969+0000 7f75f93b7640 1 --2- 172.21.15.112:0/505184223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4101870 0x7f75f4101cf0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009900 tx=0x7f75e802a410 comp rx=0 tx=0).stop 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 -- 172.21.15.112:0/505184223 shutdown_connections 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 --2- 172.21.15.112:0/505184223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f4102230 0x7f75f4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 --2- 172.21.15.112:0/505184223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4101870 0x7f75f4101cf0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 --2- 172.21.15.112:0/505184223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75f4100680 0x7f75f4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 -- 172.21.15.112:0/505184223 >> 172.21.15.112:0/505184223 conn(0x7f75f40fbbf0 msgr2=0x7f75f40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 -- 172.21.15.112:0/505184223 shutdown_connections 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 -- 172.21.15.112:0/505184223 wait complete. 2024-04-24T23:15:26.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 Processor -- start 2024-04-24T23:15:26.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.970+0000 7f75f93b7640 1 -- start start 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f93b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4100680 0x7f75f4069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f93b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f4101870 0x7f75f419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f93b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75f4102230 0x7f75f41a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f93b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75f410c430 con 0x7f75f4100680 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f93b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f75f410c2b0 con 0x7f75f4102230 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f93b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75f410c5b0 con 0x7f75f4101870 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4100680 0x7f75f4069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4100680 0x7f75f4069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52766/0 (socket says 172.21.15.112:52766) 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 -- 172.21.15.112:0/1787043974 learned_addr learned my addr 172.21.15.112:0/1787043974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f27fc640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f4101870 0x7f75f419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f37fe640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75f4102230 0x7f75f41a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 -- 172.21.15.112:0/1787043974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f4101870 msgr2=0x7f75f419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f4101870 0x7f75f419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 -- 172.21.15.112:0/1787043974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75f4102230 msgr2=0x7f75f41a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75f4102230 0x7f75f41a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 -- 172.21.15.112:0/1787043974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f41a1980 con 0x7f75f4100680 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f37fe640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75f4102230 0x7f75f41a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4100680 0x7f75f4069440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f75e400c970 tx=0x7f75e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75dbfff640 1 -- 172.21.15.112:0/1787043974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4007c10 con 0x7f75f4100680 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f93b7640 1 -- 172.21.15.112:0/1787043974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f41a1c10 con 0x7f75f4100680 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75dbfff640 1 -- 172.21.15.112:0/1787043974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75e4007db0 con 0x7f75f4100680 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75dbfff640 1 -- 172.21.15.112:0/1787043974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e400f9b0 con 0x7f75f4100680 2024-04-24T23:15:26.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.971+0000 7f75f93b7640 1 -- 172.21.15.112:0/1787043974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f41a2180 con 0x7f75f4100680 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.974+0000 7f75fc1a7740 1 Processor -- start 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.974+0000 7f75dbfff640 1 -- 172.21.15.112:0/1787043974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f75e400fb50 con 0x7f75f4100680 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.974+0000 7f75fc1a7740 1 -- start start 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a02f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 0x55cf0a034720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a037220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cf09f93670 con 0x55cf0a02f230 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cf09f3b970 con 0x55cf0a02fc20 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cf09f8f630 con 0x55cf0a034d90 2024-04-24T23:15:26.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a02f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a02f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52776/0 (socket says 172.21.15.112:52776) 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 -- 172.21.15.112:0/1106318903 learned_addr learned my addr 172.21.15.112:0/1106318903 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f27fc640 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a037220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 0x55cf0a034720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 -- 172.21.15.112:0/1106318903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 msgr2=0x55cf0a037220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a037220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75dbfff640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75dc0a2bc0 0x7f75dc0a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 -- 172.21.15.112:0/1106318903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 msgr2=0x55cf0a034720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 0x55cf0a034720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 -- 172.21.15.112:0/1106318903 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf09f5f400 con 0x55cf0a02f230 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 0x55cf0a034720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75dbfff640 1 -- 172.21.15.112:0/1787043974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f75e40cf6f0 con 0x7f75f4100680 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75dc0a2bc0 0x7f75dc0a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75f37fe640 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a02f650 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009fb0 tx=0x7f75ec010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2546f9721853a59 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.975+0000 7f75d97fa640 1 -- 172.21.15.112:0/1106318903 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec022070 con 0x55cf0a02f230 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75d97fa640 1 -- 172.21.15.112:0/1106318903 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75ec0113e0 con 0x55cf0a02f230 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75d97fa640 1 -- 172.21.15.112:0/1106318903 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec016070 con 0x55cf0a02f230 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1106318903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 msgr2=0x55cf0a02f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75f37fe640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75dc0a2bc0 0x7f75dc0a5080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01f000 tx=0x7f75ec022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a02f650 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009fb0 tx=0x7f75ec010800 comp rx=0 tx=0).stop 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1106318903 shutdown_connections 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a037220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 0x55cf0a034720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1106318903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a02f650 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1106318903 >> 172.21.15.112:0/1106318903 conn(0x55cf0a02a8a0 msgr2=0x55cf0a02aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1106318903 shutdown_connections 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.976+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1106318903 wait complete. 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 1 -- start start 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 1 Processor -- start 2024-04-24T23:15:26.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 0x55cf0a0392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a039830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a040870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cf0a037d40 con 0x55cf0a02fc20 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55cf09f4f050 con 0x55cf0a02f230 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cf0a037ec0 con 0x55cf0a034d90 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75fc1a7740 10 client.0 _ll_register_callbacks cb 0x55cf09ca6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a039830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a039830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52784/0 (socket says 172.21.15.112:52784) 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 -- 172.21.15.112:0/1375592272 learned_addr learned my addr 172.21.15.112:0/1375592272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f37fe640 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a040870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 0x55cf0a0392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 -- 172.21.15.112:0/1375592272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 msgr2=0x55cf0a040870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a040870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 -- 172.21.15.112:0/1375592272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 msgr2=0x55cf0a0392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 0x55cf0a0392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 -- 172.21.15.112:0/1375592272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf09f20dc0 con 0x55cf0a02fc20 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75f27fc640 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a039830 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009900 tx=0x7f75e8031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f4de8adaacb20 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.977+0000 7f75d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75d6ff5640 1 -- 172.21.15.112:0/1375592272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8035040 con 0x55cf0a02fc20 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75d6ff5640 1 -- 172.21.15.112:0/1375592272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f75e80050c0 con 0x55cf0a02fc20 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1375592272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75ec0113e0 con 0x55cf0a02fc20 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75d6ff5640 1 -- 172.21.15.112:0/1375592272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75e804e050 con 0x55cf0a02fc20 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 10 client.15783 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1375592272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf09ca73f0 con 0x55cf0a02fc20 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75d6ff5640 1 -- 172.21.15.112:0/1375592272 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75ec0113e0 con 0x55cf0a02fc20 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75d6ff5640 1 -- 172.21.15.112:0/1375592272 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75e80035f0 con 0x55cf0a02fc20 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 10 client.15783 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:26.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 10 client.15783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 20 client.15783 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55cf0a0450a0 0x55cf0a047560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 4 client.15783 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1375592272 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf09fa0bb0 con 0x55cf0a0450a0 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.978+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55cf0a0450a0 0x55cf0a047560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.979+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55cf0a0450a0 0x55cf0a047560 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14c51ce15ec214a4 server_cookie=8ea4bf9109a8f0c6 in_seq=0 out_seq=0 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.979+0000 7f75d6ff5640 10 client.15783 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75d6ff5640 1 -- 172.21.15.112:0/1375592272 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf09fa0bb0 con 0x55cf0a0450a0 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75d6ff5640 10 client.15783 handle_command_reply: tid=0 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 client.15783 shutdown 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1375592272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 msgr2=0x55cf0a039830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a039830 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009900 tx=0x7f75e8031ab0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1375592272 shutdown_connections 2024-04-24T23:15:26.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55cf0a0450a0 0x55cf0a047560 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a040870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a039830 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75d6ff5640 0 client.15783 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:15:26.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/1375592272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 0x55cf0a0392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1375592272 >> 172.21.15.112:0/1375592272 conn(0x55cf0a02a8a0 msgr2=0x55cf0a029b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:26.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1375592272 shutdown_connections 2024-04-24T23:15:26.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 1 -- 172.21.15.112:0/1375592272 wait complete. 2024-04-24T23:15:26.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.980+0000 7f75fc1a7740 20 client.15783 trim_cache size 0 max 16384 2024-04-24T23:15:26.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.993+0000 7f75fc1a7740 1 Processor -- start 2024-04-24T23:15:26.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.993+0000 7f75fc1a7740 1 -- start start 2024-04-24T23:15:26.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.993+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a030dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.993+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 0x55cf0a040830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.993+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a04be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:26.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.993+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41ac7e0 con 0x55cf0a02f230 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.993+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41ae320 con 0x55cf0a02fc20 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.993+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41aceb0 con 0x55cf0a034d90 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a030dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a030dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52790/0 (socket says 172.21.15.112:52790) 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 -- 172.21.15.112:0/821084235 learned_addr learned my addr 172.21.15.112:0/821084235 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 -- 172.21.15.112:0/821084235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 msgr2=0x55cf0a04be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/821084235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a04be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 -- 172.21.15.112:0/821084235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 msgr2=0x55cf0a040830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/821084235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 0x55cf0a040830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 -- 172.21.15.112:0/821084235 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf09d717d0 con 0x55cf0a02f230 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/821084235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a030dd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f75e409b8f0 tx=0x7f75e400f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ef16126f689378 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75d8ff9640 1 -- 172.21.15.112:0/821084235 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4099c90 con 0x55cf0a02f230 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75d8ff9640 1 -- 172.21.15.112:0/821084235 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75e4086040 con 0x55cf0a02f230 2024-04-24T23:15:26.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.994+0000 7f75d8ff9640 1 -- 172.21.15.112:0/821084235 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4098260 con 0x55cf0a02f230 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- 172.21.15.112:0/821084235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 msgr2=0x55cf0a030dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/821084235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a030dd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f75e409b8f0 tx=0x7f75e400f450 comp rx=0 tx=0).stop 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- 172.21.15.112:0/821084235 shutdown_connections 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/821084235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a04be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/821084235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02fc20 0x55cf0a040830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/821084235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02f230 0x55cf0a030dd0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- 172.21.15.112:0/821084235 >> 172.21.15.112:0/821084235 conn(0x55cf0a02a8a0 msgr2=0x55cf0a0306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- 172.21.15.112:0/821084235 shutdown_connections 2024-04-24T23:15:27.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- 172.21.15.112:0/821084235 wait complete. 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- start start 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 Processor -- start 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 0x55cf0a03e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a03e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a03f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41adc50 con 0x55cf0a02fc20 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41ad580 con 0x55cf0a02f230 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41aa7c0 con 0x55cf0a034d90 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.995+0000 7f75fc1a7740 10 client.0 _ll_register_callbacks cb 0x7f75e8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a03e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a03e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52802/0 (socket says 172.21.15.112:52802) 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 -- 172.21.15.112:0/3178526753 learned_addr learned my addr 172.21.15.112:0/3178526753 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:27.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 0x55cf0a03e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f37fe640 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a03f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 -- 172.21.15.112:0/3178526753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 msgr2=0x55cf0a03f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a03f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 -- 172.21.15.112:0/3178526753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 msgr2=0x55cf0a03e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 0x55cf0a03e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 -- 172.21.15.112:0/3178526753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf0a03fe00 con 0x55cf0a02fc20 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75f27fc640 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a03e800 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009850 tx=0x7f75e80045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8fb0f4b31a09fd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75d6ff5640 1 -- 172.21.15.112:0/3178526753 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8057060 con 0x55cf0a02fc20 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75fc1a7740 1 -- 172.21.15.112:0/3178526753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e4086040 con 0x55cf0a02fc20 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75d6ff5640 1 -- 172.21.15.112:0/3178526753 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f75e8003e30 con 0x55cf0a02fc20 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75d6ff5640 1 -- 172.21.15.112:0/3178526753 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75e805e050 con 0x55cf0a02fc20 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75fc1a7740 10 client.15789 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:27.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.996+0000 7f75fc1a7740 1 -- 172.21.15.112:0/3178526753 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf0a0abab0 con 0x55cf0a02fc20 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75d6ff5640 1 -- 172.21.15.112:0/3178526753 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f75e4086040 con 0x55cf0a02fc20 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75d6ff5640 1 -- 172.21.15.112:0/3178526753 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75e8004910 con 0x55cf0a02fc20 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75fc1a7740 10 client.15789 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75fc1a7740 10 client.15789 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75fc1a7740 20 client.15789 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55cf0a0ac420 0x55cf0a0ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75fc1a7740 4 client.15789 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75fc1a7740 1 -- 172.21.15.112:0/3178526753 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf09f9af00 con 0x55cf0a0ac420 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55cf0a0ac420 0x55cf0a0ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75d6ff5640 10 client.15789 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.997+0000 7f75f2ffd640 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55cf0a0ac420 0x55cf0a0ae840 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dda0a715460d23c server_cookie=a94deab4cfa2f087 in_seq=0 out_seq=0 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75d6ff5640 1 -- 172.21.15.112:0/3178526753 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf09f9af00 con 0x55cf0a0ac420 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75d6ff5640 10 client.15789 handle_command_reply: tid=0 2024-04-24T23:15:27.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 client.15789 shutdown 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 -- 172.21.15.112:0/3178526753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 msgr2=0x55cf0a03e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a03e800 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009850 tx=0x7f75e80045d0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 -- 172.21.15.112:0/3178526753 shutdown_connections 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55cf0a0ac420 0x55cf0a0ae840 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cf0a034d90 0x55cf0a03f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75d6ff5640 0 client.15789 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cf0a02fc20 0x55cf0a03e800 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 --2- 172.21.15.112:0/3178526753 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55cf0a02f230 0x55cf0a03e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 -- 172.21.15.112:0/3178526753 >> 172.21.15.112:0/3178526753 conn(0x55cf0a02a8a0 msgr2=0x55cf0a02e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 -- 172.21.15.112:0/3178526753 shutdown_connections 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 1 -- 172.21.15.112:0/3178526753 wait complete. 2024-04-24T23:15:27.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75fc1a7740 20 client.15789 trim_cache size 0 max 16384 2024-04-24T23:15:27.007 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:27.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 -- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75dc0a2bc0 msgr2=0x7f75dc0a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75dc0a2bc0 0x7f75dc0a5080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01f000 tx=0x7f75ec022000 comp rx=0 tx=0).stop 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 -- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4100680 msgr2=0x7f75f4069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4100680 0x7f75f4069440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f75e400c970 tx=0x7f75e400ce30 comp rx=0 tx=0).stop 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 -- 172.21.15.112:0/1787043974 shutdown_connections 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f75dc0a2bc0 0x7f75dc0a5080 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f75f4102230 0x7f75f41a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f4101870 0x7f75f419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 --2- 172.21.15.112:0/1787043974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75f4100680 0x7f75f4069440 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 -- 172.21.15.112:0/1787043974 >> 172.21.15.112:0/1787043974 conn(0x7f75f40fbbf0 msgr2=0x7f75f4104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.998+0000 7f75f93b7640 1 -- 172.21.15.112:0/1787043974 shutdown_connections 2024-04-24T23:15:27.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:26.999+0000 7f75f93b7640 1 -- 172.21.15.112:0/1787043974 wait complete. 2024-04-24T23:15:27.010 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:26 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2069595346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:27.011 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:15:27.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 -- 172.21.15.112:0/661984008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300102100 msgr2=0x7f33001044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 --2- 172.21.15.112:0/661984008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300102100 0x7f33001044f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009900 tx=0x7f32f002a410 comp rx=0 tx=0).stop 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 -- 172.21.15.112:0/661984008 shutdown_connections 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 --2- 172.21.15.112:0/661984008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3300107360 0x7f33001097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 --2- 172.21.15.112:0/661984008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300104a30 0x7f3300106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 --2- 172.21.15.112:0/661984008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300102100 0x7f33001044f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 -- 172.21.15.112:0/661984008 >> 172.21.15.112:0/661984008 conn(0x7f33000fbb10 msgr2=0x7f33000fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 -- 172.21.15.112:0/661984008 shutdown_connections 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.111+0000 7f3305800640 1 -- 172.21.15.112:0/661984008 wait complete. 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f3305800640 1 Processor -- start 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f3305800640 1 -- start start 2024-04-24T23:15:27.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f3305800640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102100 0x7f3300101a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f3305800640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3300104a30 0x7f33001000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f3305800640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300107360 0x7f33001005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f3305800640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f330010d450 con 0x7f3300107360 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f3305800640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f330010d2d0 con 0x7f3300102100 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f3305800640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f330010d5d0 con 0x7f3300104a30 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.112+0000 7f32ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300107360 0x7f33001005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32ff7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300107360 0x7f33001005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52810/0 (socket says 172.21.15.112:52810) 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32feffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102100 0x7f3300101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32ff7fe640 1 -- 172.21.15.112:0/207760955 learned_addr learned my addr 172.21.15.112:0/207760955 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3300104a30 0x7f33001000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32ff7fe640 1 -- 172.21.15.112:0/207760955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3300104a30 msgr2=0x7f33001000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32ff7fe640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3300104a30 0x7f33001000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32ff7fe640 1 -- 172.21.15.112:0/207760955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102100 msgr2=0x7f3300101a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32ff7fe640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102100 0x7f3300101a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32ff7fe640 1 -- 172.21.15.112:0/207760955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3300100e80 con 0x7f3300107360 2024-04-24T23:15:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3300104a30 0x7f33001000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32feffd640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102100 0x7f3300101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32ff7fe640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300107360 0x7f33001005f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f32f800eaa0 tx=0x7f32f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32e7fff640 1 -- 172.21.15.112:0/207760955 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f800cdb0 con 0x7f3300107360 2024-04-24T23:15:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f3305800640 1 -- 172.21.15.112:0/207760955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33001a1b60 con 0x7f3300107360 2024-04-24T23:15:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32e7fff640 1 -- 172.21.15.112:0/207760955 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32f8004b20 con 0x7f3300107360 2024-04-24T23:15:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f3305800640 1 -- 172.21.15.112:0/207760955 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33001a2100 con 0x7f3300107360 2024-04-24T23:15:27.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.113+0000 7f32e7fff640 1 -- 172.21.15.112:0/207760955 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8011760 con 0x7f3300107360 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f33085ee740 1 Processor -- start 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f32e7fff640 1 -- 172.21.15.112:0/207760955 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f32f8004cd0 con 0x7f3300107360 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f33085ee740 1 -- start start 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f33085ee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aa07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f33085ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 0x556291aa58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f33085ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291aa83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f33085ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556291a04670 con 0x556291aa5f20 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f33085ee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5562919ac970 con 0x556291aa03c0 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.116+0000 7f33085ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556291a00630 con 0x556291aa0db0 2024-04-24T23:15:27.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291aa83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291aa83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52816/0 (socket says 172.21.15.112:52816) 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 -- 172.21.15.112:0/150692608 learned_addr learned my addr 172.21.15.112:0/150692608 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aa07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32feffd640 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 0x556291aa58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 -- 172.21.15.112:0/150692608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 msgr2=0x556291aa58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 0x556291aa58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 -- 172.21.15.112:0/150692608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 msgr2=0x556291aa07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aa07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32e7fff640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32e80a2b90 0x7f32e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 -- 172.21.15.112:0/150692608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562919d0400 con 0x556291aa5f20 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32ff7fe640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32e80a2b90 0x7f32e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291aa83b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f32f40149f0 tx=0x7f32f400c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832846e605ee8290 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32e7fff640 1 -- 172.21.15.112:0/207760955 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f32f80cf4e0 con 0x7f3300107360 2024-04-24T23:15:27.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32e57fa640 1 -- 172.21.15.112:0/150692608 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40142d0 con 0x556291aa5f20 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32e57fa640 1 -- 172.21.15.112:0/150692608 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32f400f760 con 0x556291aa5f20 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.117+0000 7f32e57fa640 1 -- 172.21.15.112:0/150692608 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4003850 con 0x556291aa5f20 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 -- 172.21.15.112:0/150692608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 msgr2=0x556291aa83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f32ff7fe640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32e80a2b90 0x7f32e80a5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32f80ccc40 tx=0x7f32f8084cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291aa83b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f32f40149f0 tx=0x7f32f400c540 comp rx=0 tx=0).stop 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 -- 172.21.15.112:0/150692608 shutdown_connections 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291aa83b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 0x556291aa58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 --2- 172.21.15.112:0/150692608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aa07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 -- 172.21.15.112:0/150692608 >> 172.21.15.112:0/150692608 conn(0x556291a9ba30 msgr2=0x556291a9be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 -- 172.21.15.112:0/150692608 shutdown_connections 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 -- 172.21.15.112:0/150692608 wait complete. 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 -- start start 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.118+0000 7f33085ee740 1 Processor -- start 2024-04-24T23:15:27.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f33085ee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aaa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f33085ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 0x556291aaa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f33085ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291ab1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f33085ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556291aa8e70 con 0x556291aa5f20 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f33085ee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5562919c0050 con 0x556291aa03c0 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f33085ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556291aa8ff0 con 0x556291aa0db0 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f33085ee740 10 client.0 _ll_register_callbacks cb 0x556291717be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291ab1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291ab1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52830/0 (socket says 172.21.15.112:52830) 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 -- 172.21.15.112:0/3956144265 learned_addr learned my addr 172.21.15.112:0/3956144265 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32feffd640 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 0x556291aaa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aaa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 -- 172.21.15.112:0/3956144265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 msgr2=0x556291aaa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 0x556291aaa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 -- 172.21.15.112:0/3956144265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 msgr2=0x556291aaa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aaa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 -- 172.21.15.112:0/3956144265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556291991dc0 con 0x556291aa5f20 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291ab1970 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f32f400ff70 tx=0x7f32f400fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124c2c25449c47c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32e2ff5640 1 -- 172.21.15.112:0/3956144265 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4005070 con 0x556291aa5f20 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f32e2ff5640 1 -- 172.21.15.112:0/3956144265 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f32f4005210 con 0x556291aa5f20 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.119+0000 7f33085ee740 1 -- 172.21.15.112:0/3956144265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f400f760 con 0x556291aa5f20 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f32e2ff5640 1 -- 172.21.15.112:0/3956144265 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32f4035050 con 0x556291aa5f20 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f33085ee740 10 client.15801 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f33085ee740 1 -- 172.21.15.112:0/3956144265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5562917183f0 con 0x556291aa5f20 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f32e2ff5640 1 -- 172.21.15.112:0/3956144265 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32f400f760 con 0x556291aa5f20 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f32e2ff5640 1 -- 172.21.15.112:0/3956144265 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32f4030450 con 0x556291aa5f20 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f33085ee740 10 client.15801 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f33085ee740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f33085ee740 20 client.15801 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f33085ee740 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x556291ab6030 0x556291ab8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f33085ee740 4 client.15801 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:27.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f33085ee740 1 -- 172.21.15.112:0/3956144265 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556291a11bb0 con 0x556291ab6030 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.120+0000 7f32feffd640 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x556291ab6030 0x556291ab8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f32e2ff5640 10 client.15801 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f32feffd640 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x556291ab6030 0x556291ab8440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6ec1f87949affb server_cookie=ea969d06e1e7bbca in_seq=0 out_seq=0 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f32e2ff5640 1 -- 172.21.15.112:0/3956144265 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556291a11bb0 con 0x556291ab6030 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f32e2ff5640 10 client.15801 handle_command_reply: tid=0 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 client.15801 shutdown 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 -- 172.21.15.112:0/3956144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 msgr2=0x556291ab1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291ab1970 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f32f400ff70 tx=0x7f32f400fec0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 -- 172.21.15.112:0/3956144265 shutdown_connections 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x556291ab6030 0x556291ab8440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa5f20 0x556291ab1970 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa0db0 0x556291aaa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f32e2ff5640 0 client.15801 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 --2- 172.21.15.112:0/3956144265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aaa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.121+0000 7f33085ee740 1 -- 172.21.15.112:0/3956144265 >> 172.21.15.112:0/3956144265 conn(0x556291a9ba30 msgr2=0x556291a9ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.122+0000 7f33085ee740 1 -- 172.21.15.112:0/3956144265 shutdown_connections 2024-04-24T23:15:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.122+0000 7f33085ee740 1 -- 172.21.15.112:0/3956144265 wait complete. 2024-04-24T23:15:27.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.122+0000 7f33085ee740 20 client.15801 trim_cache size 0 max 16384 2024-04-24T23:15:27.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f33085ee740 1 Processor -- start 2024-04-24T23:15:27.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f33085ee740 1 -- start start 2024-04-24T23:15:27.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f33085ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa03c0 0x556291a9a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f33085ee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa0db0 0x556291ab1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f33085ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 0x556291a9e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f33085ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec00fd50 con 0x556291aa03c0 2024-04-24T23:15:27.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f33085ee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec011890 con 0x556291aa0db0 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f33085ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec010420 con 0x556291aa5f20 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa03c0 0x556291a9a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa03c0 0x556291a9a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52832/0 (socket says 172.21.15.112:52832) 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 -- 172.21.15.112:0/1586006759 learned_addr learned my addr 172.21.15.112:0/1586006759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32feffd640 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa0db0 0x556291ab1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 -- 172.21.15.112:0/1586006759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 msgr2=0x556291a9e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 0x556291a9e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 -- 172.21.15.112:0/1586006759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa0db0 msgr2=0x556291ab1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa0db0 0x556291ab1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 -- 172.21.15.112:0/1586006759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562917e27d0 con 0x556291aa03c0 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32feffd640 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa0db0 0x556291ab1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa03c0 0x556291a9a9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f32f4010010 tx=0x7f32f400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e007e2decb76a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.135+0000 7f32e4ff9640 1 -- 172.21.15.112:0/1586006759 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4026070 con 0x556291aa03c0 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f32e4ff9640 1 -- 172.21.15.112:0/1586006759 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32f400c9e0 con 0x556291aa03c0 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f32e4ff9640 1 -- 172.21.15.112:0/1586006759 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4030450 con 0x556291aa03c0 2024-04-24T23:15:27.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 -- 172.21.15.112:0/1586006759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa03c0 msgr2=0x556291a9a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa03c0 0x556291a9a9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f32f4010010 tx=0x7f32f400c8a0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 -- 172.21.15.112:0/1586006759 shutdown_connections 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 0x556291a9e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa0db0 0x556291ab1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 --2- 172.21.15.112:0/1586006759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa03c0 0x556291a9a9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 -- 172.21.15.112:0/1586006759 >> 172.21.15.112:0/1586006759 conn(0x556291a9ba30 msgr2=0x556291a9a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 -- 172.21.15.112:0/1586006759 shutdown_connections 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.136+0000 7f33085ee740 1 -- 172.21.15.112:0/1586006759 wait complete. 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 1 -- start start 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 1 Processor -- start 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa0db0 0x556291aaee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 0x556291b1a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec0111c0 con 0x556291aa0db0 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec010af0 con 0x556291aa03c0 2024-04-24T23:15:27.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec00dd30 con 0x556291aa5f20 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f33085ee740 10 client.0 _ll_register_callbacks cb 0x7f32f4014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa0db0 0x556291aaee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32feffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa0db0 0x556291aaee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52834/0 (socket says 172.21.15.112:52834) 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32feffd640 1 -- 172.21.15.112:0/1297216455 learned_addr learned my addr 172.21.15.112:0/1297216455 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 0x556291b1a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32feffd640 1 -- 172.21.15.112:0/1297216455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 msgr2=0x556291b1a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32feffd640 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 0x556291b1a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32feffd640 1 -- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 msgr2=0x556291aae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32feffd640 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32feffd640 1 -- 172.21.15.112:0/1297216455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556291b1ab20 con 0x556291aa0db0 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.137+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f32feffd640 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa0db0 0x556291aaee10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f32f00096c0 tx=0x7f32f0004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5e90da8b7705a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f32e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:27.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f32e2ff5640 1 -- 172.21.15.112:0/1297216455 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0035040 con 0x556291aa0db0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f32e2ff5640 1 -- 172.21.15.112:0/1297216455 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f32f00495f0 con 0x556291aa0db0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 1 -- 172.21.15.112:0/1297216455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f400c9e0 con 0x556291aa0db0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f32e2ff5640 1 -- 172.21.15.112:0/1297216455 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32f005a050 con 0x556291aa0db0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 10 client.15807 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 1 -- 172.21.15.112:0/1297216455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556291b1bef0 con 0x556291aa0db0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f32e2ff5640 1 -- 172.21.15.112:0/1297216455 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f32f400c9e0 con 0x556291aa0db0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f32e2ff5640 1 -- 172.21.15.112:0/1297216455 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32f0004a40 con 0x556291aa0db0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 10 client.15807 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 20 client.15807 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x556291b1c8d0 0x556291b1ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 4 client.15807 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.138+0000 7f33085ee740 1 -- 172.21.15.112:0/1297216455 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556291a0bf00 con 0x556291b1c8d0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x556291b1c8d0 0x556291b1ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:27.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f32fe7fc640 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x556291b1c8d0 0x556291b1ecf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8a78f36fee6186 server_cookie=f0609b786b7f6309 in_seq=0 out_seq=0 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f32e2ff5640 10 client.15807 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f32e2ff5640 1 -- 172.21.15.112:0/1297216455 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556291a0bf00 con 0x556291b1c8d0 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f32e2ff5640 10 client.15807 handle_command_reply: tid=0 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 client.15807 shutdown 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 -- 172.21.15.112:0/1297216455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa0db0 msgr2=0x556291aaee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa0db0 0x556291aaee10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f32f00096c0 tx=0x7f32f0004390 comp rx=0 tx=0).stop 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 -- 172.21.15.112:0/1297216455 shutdown_connections 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x556291b1c8d0 0x556291b1ecf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556291aa5f20 0x556291b1a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556291aa0db0 0x556291aaee10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f32e2ff5640 0 client.15807 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 --2- 172.21.15.112:0/1297216455 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556291aa03c0 0x556291aae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 -- 172.21.15.112:0/1297216455 >> 172.21.15.112:0/1297216455 conn(0x556291a9ba30 msgr2=0x556291a9ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 -- 172.21.15.112:0/1297216455 shutdown_connections 2024-04-24T23:15:27.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 1 -- 172.21.15.112:0/1297216455 wait complete. 2024-04-24T23:15:27.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.139+0000 7f33085ee740 20 client.15807 trim_cache size 0 max 16384 2024-04-24T23:15:27.148 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 -- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32e80a2b90 msgr2=0x7f32e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32e80a2b90 0x7f32e80a5050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f32f80ccc40 tx=0x7f32f8084cf0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 -- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300107360 msgr2=0x7f33001005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300107360 0x7f33001005f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f32f800eaa0 tx=0x7f32f800ef60 comp rx=0 tx=0).stop 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 -- 172.21.15.112:0/207760955 shutdown_connections 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32e80a2b90 0x7f32e80a5050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3300107360 0x7f33001005f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3300104a30 0x7f33001000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 --2- 172.21.15.112:0/207760955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3300102100 0x7f3300101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 -- 172.21.15.112:0/207760955 >> 172.21.15.112:0/207760955 conn(0x7f33000fbb10 msgr2=0x7f3300102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 -- 172.21.15.112:0/207760955 shutdown_connections 2024-04-24T23:15:27.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:27.140+0000 7f3305800640 1 -- 172.21.15.112:0/207760955 wait complete. 2024-04-24T23:15:27.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:26 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2069595346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:27.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:26 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2069595346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:15:28.157 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:15:28.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:15:28.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:15:28.178 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:27 smithi112 ceph-mon[29186]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:28.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:27 smithi187 ceph-mon[33292]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- 172.21.15.112:0/1358532379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808101870 msgr2=0x7f8808101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 --2- 172.21.15.112:0/1358532379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808101870 0x7f8808101cf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009900 tx=0x7f87f802a410 comp rx=0 tx=0).stop 2024-04-24T23:15:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- 172.21.15.112:0/1358532379 shutdown_connections 2024-04-24T23:15:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 --2- 172.21.15.112:0/1358532379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8808102230 0x7f8808108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 --2- 172.21.15.112:0/1358532379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808101870 0x7f8808101cf0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 --2- 172.21.15.112:0/1358532379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8808100680 0x7f8808100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- 172.21.15.112:0/1358532379 >> 172.21.15.112:0/1358532379 conn(0x7f88080fbbf0 msgr2=0x7f88080fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- 172.21.15.112:0/1358532379 shutdown_connections 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- 172.21.15.112:0/1358532379 wait complete. 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 Processor -- start 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- start start 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8808100680 0x7f8808069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8808101870 0x7f880819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808102230 0x7f880819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f880810c430 con 0x7f8808102230 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f880810c2b0 con 0x7f8808100680 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.262+0000 7f881055a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f880810c5b0 con 0x7f8808101870 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808102230 0x7f880819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808102230 0x7f880819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52852/0 (socket says 172.21.15.112:52852) 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 -- 172.21.15.112:0/3034497610 learned_addr learned my addr 172.21.15.112:0/3034497610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880e2f6640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8808100680 0x7f8808069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880daf5640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8808101870 0x7f880819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 -- 172.21.15.112:0/3034497610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8808101870 msgr2=0x7f880819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8808101870 0x7f880819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 -- 172.21.15.112:0/3034497610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8808100680 msgr2=0x7f8808069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8808100680 0x7f8808069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 -- 172.21.15.112:0/3034497610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880819f7f0 con 0x7f8808102230 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f880eaf7640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808102230 0x7f880819ef30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8800009fd0 tx=0x7f880000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f87ff7fe640 1 -- 172.21.15.112:0/3034497610 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800017070 con 0x7f8808102230 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f87ff7fe640 1 -- 172.21.15.112:0/3034497610 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8800004020 con 0x7f8808102230 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f881055a640 1 -- 172.21.15.112:0/3034497610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880819fae0 con 0x7f8808102230 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f87ff7fe640 1 -- 172.21.15.112:0/3034497610 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88000106c0 con 0x7f8808102230 2024-04-24T23:15:28.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.263+0000 7f881055a640 1 -- 172.21.15.112:0/3034497610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880819ffb0 con 0x7f8808102230 2024-04-24T23:15:28.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.265+0000 7f87ff7fe640 1 -- 172.21.15.112:0/3034497610 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f88000041d0 con 0x7f8808102230 2024-04-24T23:15:28.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.266+0000 7f8813266740 1 Processor -- start 2024-04-24T23:15:28.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.266+0000 7f87ff7fe640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87f40a2b90 0x7f87f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.266+0000 7f87ff7fe640 1 -- 172.21.15.112:0/3034497610 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f88000cf020 con 0x7f8808102230 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.266+0000 7f880daf5640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87f40a2b90 0x7f87f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.266+0000 7f8813266740 1 -- start start 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.266+0000 7f8813266740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3cf45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f8813266740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cf9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f8813266740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3cfc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f8813266740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5643c3c532c0 con 0x5643c3cf4b90 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f8813266740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5643c3bfb910 con 0x5643c3cf41a0 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f8813266740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5643c3c4f280 con 0x5643c3cf9d00 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cf9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cf9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52858/0 (socket says 172.21.15.112:52858) 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 -- 172.21.15.112:0/1949710731 learned_addr learned my addr 172.21.15.112:0/1949710731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87f40a2b90 0x7f87f40a5050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006010 tx=0x7f87f8031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880e2f6640 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3cf45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880eaf7640 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3cfc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 -- 172.21.15.112:0/1949710731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 msgr2=0x5643c3cfc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3cfc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 -- 172.21.15.112:0/1949710731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 msgr2=0x5643c3cf45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3cf45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 -- 172.21.15.112:0/1949710731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c3c0cfb0 con 0x5643c3cf4b90 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880eaf7640 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3cfc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f880daf5640 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cf9690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87f8033000 tx=0x7f87f803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e612742772c9880d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f87fcbfe640 1 -- 172.21.15.112:0/1949710731 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f804d070 con 0x5643c3cf4b90 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f87fcbfe640 1 -- 172.21.15.112:0/1949710731 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f87f8004440 con 0x5643c3cf4b90 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.267+0000 7f87fcbfe640 1 -- 172.21.15.112:0/1949710731 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8035040 con 0x5643c3cf4b90 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 -- 172.21.15.112:0/1949710731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 msgr2=0x5643c3cf9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cf9690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f87f8033000 tx=0x7f87f803d000 comp rx=0 tx=0).stop 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 -- 172.21.15.112:0/1949710731 shutdown_connections 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3cfc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cf9690 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 --2- 172.21.15.112:0/1949710731 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3cf45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 -- 172.21.15.112:0/1949710731 >> 172.21.15.112:0/1949710731 conn(0x5643c3cb7950 msgr2=0x5643c3ceb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 -- 172.21.15.112:0/1949710731 shutdown_connections 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.268+0000 7f8813266740 1 -- 172.21.15.112:0/1949710731 wait complete. 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 1 -- start start 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 1 Processor -- start 2024-04-24T23:15:28.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3cfe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cfead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3d05b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5643c3cfcc50 con 0x5643c3cf4b90 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5643c3c0ee90 con 0x5643c3cf41a0 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5643c3cfcdd0 con 0x5643c3cf9d00 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f8813266740 10 client.0 _ll_register_callbacks cb 0x5643c3c4a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880daf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cfead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880daf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cfead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52870/0 (socket says 172.21.15.112:52870) 2024-04-24T23:15:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880daf5640 1 -- 172.21.15.112:0/3758716847 learned_addr learned my addr 172.21.15.112:0/3758716847 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880e2f6640 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3cfe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880eaf7640 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3d05b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880daf5640 1 -- 172.21.15.112:0/3758716847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 msgr2=0x5643c3d05b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880daf5640 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3d05b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880daf5640 1 -- 172.21.15.112:0/3758716847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 msgr2=0x5643c3cfe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880daf5640 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3cfe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.269+0000 7f880daf5640 1 -- 172.21.15.112:0/3758716847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c3a2f7e0 con 0x5643c3cf4b90 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f880daf5640 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cfead0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f87f8031570 tx=0x7f87f8049680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc770a8404feb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f87f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f87f1ffb640 1 -- 172.21.15.112:0/3758716847 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f805a070 con 0x5643c3cf4b90 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 1 -- 172.21.15.112:0/3758716847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f8004440 con 0x5643c3cf4b90 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f87f1ffb640 1 -- 172.21.15.112:0/3758716847 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f87f8049cf0 con 0x5643c3cf4b90 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f87f1ffb640 1 -- 172.21.15.112:0/3758716847 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f87f8066050 con 0x5643c3cf4b90 2024-04-24T23:15:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 10 client.15819 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 1 -- 172.21.15.112:0/3758716847 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643c3a8ff70 con 0x5643c3cf4b90 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f87f1ffb640 1 -- 172.21.15.112:0/3758716847 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f87f8004440 con 0x5643c3cf4b90 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f87f1ffb640 1 -- 172.21.15.112:0/3758716847 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f87f8052860 con 0x5643c3cf4b90 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 10 client.15819 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 20 client.15819 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5643c3d09c70 0x5643c3d0c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 4 client.15819 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.270+0000 7f8813266740 1 -- 172.21.15.112:0/3758716847 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c3c275a0 con 0x5643c3d09c70 2024-04-24T23:15:28.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.271+0000 7f880e2f6640 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5643c3d09c70 0x5643c3d0c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.271+0000 7f880e2f6640 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5643c3d09c70 0x5643c3d0c060 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4214e7c1cca44cd0 server_cookie=c70fd91f26b2fca3 in_seq=0 out_seq=0 2024-04-24T23:15:28.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.271+0000 7f87f1ffb640 10 client.15819 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:28.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f87f1ffb640 1 -- 172.21.15.112:0/3758716847 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643c3c275a0 con 0x5643c3d09c70 2024-04-24T23:15:28.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f87f1ffb640 10 client.15819 handle_command_reply: tid=0 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 client.15819 shutdown 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 -- 172.21.15.112:0/3758716847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 msgr2=0x5643c3cfead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cfead0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f87f8031570 tx=0x7f87f8049680 comp rx=0 tx=0).stop 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 -- 172.21.15.112:0/3758716847 shutdown_connections 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5643c3d09c70 0x5643c3d0c060 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3cf9d00 0x5643c3d05b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f87f1ffb640 0 client.15819 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf4b90 0x5643c3cfead0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 --2- 172.21.15.112:0/3758716847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3cfe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 -- 172.21.15.112:0/3758716847 >> 172.21.15.112:0/3758716847 conn(0x5643c3cb7950 msgr2=0x5643c3cea950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 -- 172.21.15.112:0/3758716847 shutdown_connections 2024-04-24T23:15:28.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 1 -- 172.21.15.112:0/3758716847 wait complete. 2024-04-24T23:15:28.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.272+0000 7f8813266740 20 client.15819 trim_cache size 0 max 16384 2024-04-24T23:15:28.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f8813266740 1 Processor -- start 2024-04-24T23:15:28.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f8813266740 1 -- start start 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f8813266740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 0x5643c3ceb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f8813266740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cb7950 0x5643c3d05ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f8813266740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3ceebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f8813266740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88081a8460 con 0x5643c3cb7950 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f8813266740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88081a9fa0 con 0x5643c3cf41a0 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f8813266740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88081a8b30 con 0x5643c3ceb8f0 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cb7950 0x5643c3d05ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cb7950 0x5643c3d05ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52886/0 (socket says 172.21.15.112:52886) 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 -- 172.21.15.112:0/4188706474 learned_addr learned my addr 172.21.15.112:0/4188706474 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 -- 172.21.15.112:0/4188706474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 msgr2=0x5643c3ceb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 --2- 172.21.15.112:0/4188706474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 0x5643c3ceb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 -- 172.21.15.112:0/4188706474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 msgr2=0x5643c3ceebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 --2- 172.21.15.112:0/4188706474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3ceebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 -- 172.21.15.112:0/4188706474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c3927170 con 0x5643c3cb7950 2024-04-24T23:15:28.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f880daf5640 1 --2- 172.21.15.112:0/4188706474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cb7950 0x5643c3d05ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f87f8052c00 tx=0x7f87f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7f12aacf4c678 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f87f3fff640 1 -- 172.21.15.112:0/4188706474 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8040070 con 0x5643c3cb7950 2024-04-24T23:15:28.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.290+0000 7f87f3fff640 1 -- 172.21.15.112:0/4188706474 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f87f80496f0 con 0x5643c3cb7950 2024-04-24T23:15:28.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f87f3fff640 1 -- 172.21.15.112:0/4188706474 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f805a040 con 0x5643c3cb7950 2024-04-24T23:15:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 -- 172.21.15.112:0/4188706474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cb7950 msgr2=0x5643c3d05ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 --2- 172.21.15.112:0/4188706474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cb7950 0x5643c3d05ad0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f87f8052c00 tx=0x7f87f8000bc0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 -- 172.21.15.112:0/4188706474 shutdown_connections 2024-04-24T23:15:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 --2- 172.21.15.112:0/4188706474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cf41a0 0x5643c3ceebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 --2- 172.21.15.112:0/4188706474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cb7950 0x5643c3d05ad0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 --2- 172.21.15.112:0/4188706474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 0x5643c3ceb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 -- 172.21.15.112:0/4188706474 >> 172.21.15.112:0/4188706474 conn(0x5643c3cf5790 msgr2=0x5643c3cee4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 -- 172.21.15.112:0/4188706474 shutdown_connections 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 -- 172.21.15.112:0/4188706474 wait complete. 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.291+0000 7f8813266740 1 -- start start 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f8813266740 1 Processor -- start 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f8813266740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cb7950 0x5643c3d03750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f8813266740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 0x5643c3d03c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f8813266740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf41a0 0x5643c3d04ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f8813266740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88081a98d0 con 0x5643c3cf41a0 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f8813266740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f88081a9200 con 0x5643c3cb7950 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f8813266740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88081a6440 con 0x5643c3ceb8f0 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f8813266740 10 client.0 _ll_register_callbacks cb 0x7f8804004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf41a0 0x5643c3d04ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf41a0 0x5643c3d04ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52892/0 (socket says 172.21.15.112:52892) 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 -- 172.21.15.112:0/1717831668 learned_addr learned my addr 172.21.15.112:0/1717831668 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880daf5640 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 0x5643c3d03c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880e2f6640 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cb7950 0x5643c3d03750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 -- 172.21.15.112:0/1717831668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 msgr2=0x5643c3d03c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 0x5643c3d03c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 -- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cb7950 msgr2=0x5643c3d03750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cb7950 0x5643c3d03750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 -- 172.21.15.112:0/1717831668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c3d05290 con 0x5643c3cf41a0 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880e2f6640 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cb7950 0x5643c3d03750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f880eaf7640 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf41a0 0x5643c3d04ca0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f88000cc730 tx=0x7f880009b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683f11e26863c626 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f87f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:28.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.292+0000 7f87f1ffb640 1 -- 172.21.15.112:0/1717831668 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800020040 con 0x5643c3cf41a0 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f87f1ffb640 1 -- 172.21.15.112:0/1717831668 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f8800097bb0 con 0x5643c3cf41a0 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 1 -- 172.21.15.112:0/1717831668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f80496f0 con 0x5643c3cf41a0 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f87f1ffb640 1 -- 172.21.15.112:0/1717831668 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f880009f840 con 0x5643c3cf41a0 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 10 client.15825 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 1 -- 172.21.15.112:0/1717831668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643c3d70650 con 0x5643c3cf41a0 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f87f1ffb640 1 -- 172.21.15.112:0/1717831668 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f87f80496f0 con 0x5643c3cf41a0 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f87f1ffb640 1 -- 172.21.15.112:0/1717831668 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f880009ab00 con 0x5643c3cf41a0 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 10 client.15825 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 20 client.15825 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5643c3cf4b90 0x5643c3d73060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 4 client.15825 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:15:28.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f8813266740 1 -- 172.21.15.112:0/1717831668 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5643c3c60800 con 0x5643c3cf4b90 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.293+0000 7f880e2f6640 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5643c3cf4b90 0x5643c3d73060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.294+0000 7f87f1ffb640 10 client.15825 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.294+0000 7f880e2f6640 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5643c3cf4b90 0x5643c3d73060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1ecfa2392c0346 server_cookie=b178c7d21b2cfbef in_seq=0 out_seq=0 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f87f1ffb640 1 -- 172.21.15.112:0/1717831668 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5643c3c60800 con 0x5643c3cf4b90 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f87f1ffb640 10 client.15825 handle_command_reply: tid=0 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 client.15825 shutdown 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 -- 172.21.15.112:0/1717831668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf41a0 msgr2=0x5643c3d04ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf41a0 0x5643c3d04ca0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f88000cc730 tx=0x7f880009b140 comp rx=0 tx=0).stop 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 -- 172.21.15.112:0/1717831668 shutdown_connections 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5643c3cf4b90 0x5643c3d73060 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5643c3cf41a0 0x5643c3d04ca0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5643c3ceb8f0 0x5643c3d03c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 --2- 172.21.15.112:0/1717831668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5643c3cb7950 0x5643c3d03750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 -- 172.21.15.112:0/1717831668 >> 172.21.15.112:0/1717831668 conn(0x5643c3cf5790 msgr2=0x5643c3ceed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f87f1ffb640 0 client.15825 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 -- 172.21.15.112:0/1717831668 shutdown_connections 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 1 -- 172.21.15.112:0/1717831668 wait complete. 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.295+0000 7f8813266740 20 client.15825 trim_cache size 0 max 16384 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "0027eafc-ccb1-4ae9-9edb-cdee99640e96", 2024-04-24T23:15:28.303 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-04-24T23:15:28.304 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:15:28.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 -- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87f40a2b90 msgr2=0x7f87f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87f40a2b90 0x7f87f40a5050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006010 tx=0x7f87f8031640 comp rx=0 tx=0).stop 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 -- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808102230 msgr2=0x7f880819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808102230 0x7f880819ef30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8800009fd0 tx=0x7f880000b260 comp rx=0 tx=0).stop 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 -- 172.21.15.112:0/3034497610 shutdown_connections 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87f40a2b90 0x7f87f40a5050 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8808102230 0x7f880819ef30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8808101870 0x7f880819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 --2- 172.21.15.112:0/3034497610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8808100680 0x7f8808069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 -- 172.21.15.112:0/3034497610 >> 172.21.15.112:0/3034497610 conn(0x7f88080fbbf0 msgr2=0x7f8808104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 -- 172.21.15.112:0/3034497610 shutdown_connections 2024-04-24T23:15:28.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.296+0000 7f881055a640 1 -- 172.21.15.112:0/3034497610 wait complete. 2024-04-24T23:15:28.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:15:28.308 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:15:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:27 smithi150 ceph-mon[32247]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:28.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 -- 172.21.15.112:0/3968105804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960100680 msgr2=0x7fd960100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 --2- 172.21.15.112:0/3968105804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960100680 0x7fd960100aa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd950009900 tx=0x7fd95002a410 comp rx=0 tx=0).stop 2024-04-24T23:15:28.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 -- 172.21.15.112:0/3968105804 shutdown_connections 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 --2- 172.21.15.112:0/3968105804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd960102230 0x7fd960108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 --2- 172.21.15.112:0/3968105804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd960101870 0x7fd960101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 --2- 172.21.15.112:0/3968105804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960100680 0x7fd960100aa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 -- 172.21.15.112:0/3968105804 >> 172.21.15.112:0/3968105804 conn(0x7fd9600fbbf0 msgr2=0x7fd9600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 -- 172.21.15.112:0/3968105804 shutdown_connections 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 -- 172.21.15.112:0/3968105804 wait complete. 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 Processor -- start 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.449+0000 7fd9663d5640 1 -- start start 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd9663d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd960100680 0x7fd960197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd9663d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960101870 0x7fd960198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd9663d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd960102230 0x7fd96019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd9663d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd96010c450 con 0x7fd960101870 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd9663d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd96010c2d0 con 0x7fd960100680 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd9663d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd96010c5d0 con 0x7fd960102230 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960101870 0x7fd960198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960101870 0x7fd960198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52900/0 (socket says 172.21.15.112:52900) 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 -- 172.21.15.112:0/2071819350 learned_addr learned my addr 172.21.15.112:0/2071819350 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95ffff640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd960100680 0x7fd960197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd964972640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd960102230 0x7fd96019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 -- 172.21.15.112:0/2071819350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd960102230 msgr2=0x7fd96019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd960102230 0x7fd96019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 -- 172.21.15.112:0/2071819350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd960100680 msgr2=0x7fd960197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd960100680 0x7fd960197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 -- 172.21.15.112:0/2071819350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96019f810 con 0x7fd960101870 2024-04-24T23:15:28.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960101870 0x7fd960198130 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd95400c970 tx=0x7fd95400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.450+0000 7fd95d7fa640 1 -- 172.21.15.112:0/2071819350 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954007bb0 con 0x7fd960101870 2024-04-24T23:15:28.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.451+0000 7fd95d7fa640 1 -- 172.21.15.112:0/2071819350 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd954007d50 con 0x7fd960101870 2024-04-24T23:15:28.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.451+0000 7fd9663d5640 1 -- 172.21.15.112:0/2071819350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96019fb00 con 0x7fd960101870 2024-04-24T23:15:28.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.451+0000 7fd95d7fa640 1 -- 172.21.15.112:0/2071819350 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95400f4b0 con 0x7fd960101870 2024-04-24T23:15:28.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.451+0000 7fd9663d5640 1 -- 172.21.15.112:0/2071819350 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9601a0000 con 0x7fd960101870 2024-04-24T23:15:28.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd95d7fa640 1 -- 172.21.15.112:0/2071819350 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd9540046c0 con 0x7fd960101870 2024-04-24T23:15:28.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd95d7fa640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9480a2b90 0x7fd9480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd9691c4740 1 Processor -- start 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd9691c4740 1 -- start start 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd95ffff640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9480a2b90 0x7fd9480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd95d7fa640 1 -- 172.21.15.112:0/2071819350 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd9540cf1f0 con 0x7fd960101870 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50baca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bacff0 0x55af50baf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.453+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bafc20 0x55af50bb4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd9691c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55af50b106b0 con 0x55af50bafc20 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd9691c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55af50ab8880 con 0x55af50bac600 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd9691c4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55af50b0c670 con 0x55af50bacff0 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bafc20 0x55af50bb4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bafc20 0x55af50bb4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52916/0 (socket says 172.21.15.112:52916) 2024-04-24T23:15:28.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 -- 172.21.15.112:0/92053024 learned_addr learned my addr 172.21.15.112:0/92053024 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd95ffff640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9480a2b90 0x7fd9480a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd950031570 tx=0x7fd950031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd95ffff640 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50baca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bacff0 0x55af50baf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 -- 172.21.15.112:0/92053024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bacff0 msgr2=0x55af50baf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bacff0 0x55af50baf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 -- 172.21.15.112:0/92053024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 msgr2=0x55af50baca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50baca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 -- 172.21.15.112:0/92053024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af508ee7d0 con 0x55af50bafc20 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd95ffff640 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50baca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bacff0 0x55af50baf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd964972640 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bafc20 0x55af50bb4480 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd958009ee0 tx=0x7fd958010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc70a04a3a979595 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd9467fc640 1 -- 172.21.15.112:0/92053024 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958016070 con 0x55af50bafc20 2024-04-24T23:15:28.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd9467fc640 1 -- 172.21.15.112:0/92053024 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd958004d20 con 0x55af50bafc20 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.454+0000 7fd9467fc640 1 -- 172.21.15.112:0/92053024 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958012870 con 0x55af50bafc20 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 -- 172.21.15.112:0/92053024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bafc20 msgr2=0x55af50bb4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bafc20 0x55af50bb4480 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd958009ee0 tx=0x7fd958010740 comp rx=0 tx=0).stop 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 -- 172.21.15.112:0/92053024 shutdown_connections 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bafc20 0x55af50bb4480 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bacff0 0x55af50baf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 --2- 172.21.15.112:0/92053024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50baca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 -- 172.21.15.112:0/92053024 >> 172.21.15.112:0/92053024 conn(0x55af50ba78b0 msgr2=0x55af50ba7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 -- 172.21.15.112:0/92053024 shutdown_connections 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 -- 172.21.15.112:0/92053024 wait complete. 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.455+0000 7fd9691c4740 1 -- start start 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd9691c4740 1 Processor -- start 2024-04-24T23:15:28.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bac600 0x55af50bb6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bacff0 0x55af50bb6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 0x55af50bbdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd9691c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55af50bb4d70 con 0x55af50bac600 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd9691c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55af50acbf50 con 0x55af50bacff0 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd9691c4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55af50bb4ef0 con 0x55af50bafc20 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd9691c4740 10 client.0 _ll_register_callbacks cb 0x55af50b07000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bac600 0x55af50bb6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bac600 0x55af50bb6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52920/0 (socket says 172.21.15.112:52920) 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 -- 172.21.15.112:0/2070697668 learned_addr learned my addr 172.21.15.112:0/2070697668 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd964972640 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 0x55af50bbdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bacff0 0x55af50bb6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 -- 172.21.15.112:0/2070697668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 msgr2=0x55af50bbdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 0x55af50bbdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 -- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bacff0 msgr2=0x55af50bb6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bacff0 0x55af50bb6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 -- 172.21.15.112:0/2070697668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af508243f0 con 0x55af50bac600 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bacff0 0x55af50bb6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd95ffff640 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bac600 0x55af50bb6600 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd950009900 tx=0x7fd95002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a7073778fea67 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.456+0000 7fd93f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd93f7fe640 1 -- 172.21.15.112:0/2070697668 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950035040 con 0x55af50bac600 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd93f7fe640 1 -- 172.21.15.112:0/2070697668 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd950003a00 con 0x55af50bac600 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 1 -- 172.21.15.112:0/2070697668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd958004d20 con 0x55af50bac600 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd93f7fe640 1 -- 172.21.15.112:0/2070697668 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd950060050 con 0x55af50bac600 2024-04-24T23:15:28.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 10 client.15837 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 1 -- 172.21.15.112:0/2070697668 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af507e40f0 con 0x55af50bac600 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd93f7fe640 1 -- 172.21.15.112:0/2070697668 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd958004d20 con 0x55af50bac600 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd93f7fe640 1 -- 172.21.15.112:0/2070697668 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd95004fcb0 con 0x55af50bac600 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 10 client.15837 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 20 client.15837 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55af50bc2470 0x55af50bc4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 4 client.15837 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd9691c4740 1 -- 172.21.15.112:0/2070697668 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af50b1dbf0 con 0x55af50bc2470 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.457+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55af50bc2470 0x55af50bc4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.458+0000 7fd93f7fe640 10 client.15837 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.458+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55af50bc2470 0x55af50bc4900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95e7988d5804e6b6 server_cookie=acb28d7cca0659d2 in_seq=0 out_seq=0 2024-04-24T23:15:28.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.458+0000 7fd93f7fe640 1 -- 172.21.15.112:0/2070697668 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af50b1dbf0 con 0x55af50bc2470 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.458+0000 7fd93f7fe640 10 client.15837 handle_command_reply: tid=0 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.458+0000 7fd9691c4740 1 client.15837 shutdown 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 -- 172.21.15.112:0/2070697668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bac600 msgr2=0x55af50bb6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bac600 0x55af50bb6600 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd950009900 tx=0x7fd95002a410 comp rx=0 tx=0).stop 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 -- 172.21.15.112:0/2070697668 shutdown_connections 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55af50bc2470 0x55af50bc4900 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd93f7fe640 0 client.15837 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 0x55af50bbdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bacff0 0x55af50bb6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 --2- 172.21.15.112:0/2070697668 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bac600 0x55af50bb6600 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 -- 172.21.15.112:0/2070697668 >> 172.21.15.112:0/2070697668 conn(0x55af50ba78b0 msgr2=0x55af50ba6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 -- 172.21.15.112:0/2070697668 shutdown_connections 2024-04-24T23:15:28.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 1 -- 172.21.15.112:0/2070697668 wait complete. 2024-04-24T23:15:28.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.459+0000 7fd9691c4740 20 client.15837 trim_cache size 0 max 16384 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.479+0000 7fd9691c4740 1 Processor -- start 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.479+0000 7fd9691c4740 1 -- start start 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.479+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50bb6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 0x55af50bb6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd9691c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9601a83f0 con 0x55af50bacff0 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd9691c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9601a9f30 con 0x55af50bac600 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd9691c4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9601a8ac0 con 0x55af50bafc20 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52934/0 (socket says 172.21.15.112:52934) 2024-04-24T23:15:28.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 -- 172.21.15.112:0/506251130 learned_addr learned my addr 172.21.15.112:0/506251130 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95ffff640 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50bb6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd964972640 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 0x55af50bb6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 -- 172.21.15.112:0/506251130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 msgr2=0x55af50bb6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 0x55af50bb6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 -- 172.21.15.112:0/506251130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 msgr2=0x55af50bb6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50bb6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 -- 172.21.15.112:0/506251130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af50b99090 con 0x55af50bacff0 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95ffff640 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50bb6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbdb40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd9540ccc70 tx=0x7fd954004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59f4f6585b0ab54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd945ffb640 1 -- 172.21.15.112:0/506251130 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95401d040 con 0x55af50bacff0 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd945ffb640 1 -- 172.21.15.112:0/506251130 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd95409b660 con 0x55af50bacff0 2024-04-24T23:15:28.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.480+0000 7fd945ffb640 1 -- 172.21.15.112:0/506251130 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9540999f0 con 0x55af50bacff0 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 -- 172.21.15.112:0/506251130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 msgr2=0x55af50bbdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbdb40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd9540ccc70 tx=0x7fd954004b00 comp rx=0 tx=0).stop 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 -- 172.21.15.112:0/506251130 shutdown_connections 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bafc20 0x55af50bb6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbdb40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 --2- 172.21.15.112:0/506251130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bac600 0x55af50bb6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 -- 172.21.15.112:0/506251130 >> 172.21.15.112:0/506251130 conn(0x55af50ba78b0 msgr2=0x55af50ba7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 -- 172.21.15.112:0/506251130 shutdown_connections 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 -- 172.21.15.112:0/506251130 wait complete. 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.481+0000 7fd9691c4740 1 -- start start 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd9691c4740 1 Processor -- start 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bac600 0x55af50bbaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd9691c4740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bafc20 0x55af50bbc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd9691c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9601a9860 con 0x55af50bacff0 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd9691c4740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9601a9190 con 0x55af50bafc20 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd9691c4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9601a63d0 con 0x55af50bac600 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd9691c4740 10 client.0 _ll_register_callbacks cb 0x7fd950009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52936/0 (socket says 172.21.15.112:52936) 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 -- 172.21.15.112:0/1372663885 learned_addr learned my addr 172.21.15.112:0/1372663885 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95ffff640 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bac600 0x55af50bbaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd964972640 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bafc20 0x55af50bbc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 -- 172.21.15.112:0/1372663885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bac600 msgr2=0x55af50bbaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bac600 0x55af50bbaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 -- 172.21.15.112:0/1372663885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bafc20 msgr2=0x55af50bbc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bafc20 0x55af50bbc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 -- 172.21.15.112:0/1372663885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af50bbca10 con 0x55af50bacff0 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd95f7fe640 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbb410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd954004ab0 tx=0x7fd9540a42d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38706b41d796515 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd93f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.482+0000 7fd93f7fe640 1 -- 172.21.15.112:0/1372663885 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9540b9070 con 0x55af50bacff0 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd93f7fe640 1 -- 172.21.15.112:0/1372663885 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd9540afb80 con 0x55af50bacff0 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 1 -- 172.21.15.112:0/1372663885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd95409b660 con 0x55af50bacff0 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd93f7fe640 1 -- 172.21.15.112:0/1372663885 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd954082050 con 0x55af50bacff0 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 10 client.15843 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 1 -- 172.21.15.112:0/1372663885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55af50bbdd90 con 0x55af50bacff0 2024-04-24T23:15:28.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd93f7fe640 1 -- 172.21.15.112:0/1372663885 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd95409b660 con 0x55af50bacff0 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd93f7fe640 1 -- 172.21.15.112:0/1372663885 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9540b42b0 con 0x55af50bacff0 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 10 client.15843 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 20 client.15843 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55af50c29060 0x55af50c2b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 4 client.15843 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd9691c4740 1 -- 172.21.15.112:0/1372663885 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55af50b17f40 con 0x55af50c29060 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.483+0000 7fd95ffff640 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55af50c29060 0x55af50c2b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd95ffff640 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55af50c29060 0x55af50c2b480 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c561971f7e8e288f server_cookie=f3c5828188e3ca1e in_seq=0 out_seq=0 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd93f7fe640 10 client.15843 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd93f7fe640 1 -- 172.21.15.112:0/1372663885 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55af50b17f40 con 0x55af50c29060 2024-04-24T23:15:28.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd93f7fe640 10 client.15843 handle_command_reply: tid=0 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 client.15843 shutdown 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 -- 172.21.15.112:0/1372663885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 msgr2=0x55af50bbb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbb410 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd954004ab0 tx=0x7fd9540a42d0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 -- 172.21.15.112:0/1372663885 shutdown_connections 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55af50c29060 0x55af50c2b480 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55af50bafc20 0x55af50bbc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55af50bacff0 0x55af50bbb410 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd93f7fe640 0 client.15843 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 --2- 172.21.15.112:0/1372663885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af50bac600 0x55af50bbaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 -- 172.21.15.112:0/1372663885 >> 172.21.15.112:0/1372663885 conn(0x55af50ba78b0 msgr2=0x55af50bb6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 -- 172.21.15.112:0/1372663885 shutdown_connections 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 1 -- 172.21.15.112:0/1372663885 wait complete. 2024-04-24T23:15:28.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.484+0000 7fd9691c4740 20 client.15843 trim_cache size 0 max 16384 2024-04-24T23:15:28.494 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:15:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:15:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-04-24T23:15:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "0027eafc-ccb1-4ae9-9edb-cdee99640e96": { 2024-04-24T23:15:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-04-24T23:15:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "0027eafc-ccb1-4ae9-9edb-cdee99640e96", 2024-04-24T23:15:28.494 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-04-24T23:15:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:15:28.495 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:15:28.495 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:15:28.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 -- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9480a2b90 msgr2=0x7fd9480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9480a2b90 0x7fd9480a5050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd950031570 tx=0x7fd950031640 comp rx=0 tx=0).stop 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 -- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960101870 msgr2=0x7fd960198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960101870 0x7fd960198130 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd95400c970 tx=0x7fd95400ce30 comp rx=0 tx=0).stop 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 -- 172.21.15.112:0/2071819350 shutdown_connections 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9480a2b90 0x7fd9480a5050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd960102230 0x7fd96019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd960101870 0x7fd960198130 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 --2- 172.21.15.112:0/2071819350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd960100680 0x7fd960197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 -- 172.21.15.112:0/2071819350 >> 172.21.15.112:0/2071819350 conn(0x7fd9600fbbf0 msgr2=0x7fd9600688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 -- 172.21.15.112:0/2071819350 shutdown_connections 2024-04-24T23:15:28.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:28.485+0000 7fd9663d5640 1 -- 172.21.15.112:0/2071819350 wait complete. 2024-04-24T23:15:28.497 INFO:tasks.cephfs.filesystem:scrub status for tag:0027eafc-ccb1-4ae9-9edb-cdee99640e96 - {'path': '/', 'tag': '0027eafc-ccb1-4ae9-9edb-cdee99640e96', 'options': 'recursive,force'} 2024-04-24T23:15:29.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:28 smithi112 ceph-mon[29186]: scrub queued for path: / 2024-04-24T23:15:29.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:28 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:29.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:28 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:15:29.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:28 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:28 smithi187 ceph-mon[33292]: scrub queued for path: / 2024-04-24T23:15:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:28 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:28 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:15:29.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:28 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:28 smithi150 ceph-mon[32247]: scrub queued for path: / 2024-04-24T23:15:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:28 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:28 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:15:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:28 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:15:30.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:29 smithi187 ceph-mon[33292]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-04-24T23:15:30.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:29 smithi112 ceph-mon[29186]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-04-24T23:15:30.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:29 smithi150 ceph-mon[32247]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-04-24T23:15:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:31 smithi187 ceph-mon[33292]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-24T23:15:32.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:31 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:15:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:31 smithi112 ceph-mon[29186]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-24T23:15:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:31 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:15:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:31 smithi150 ceph-mon[32247]: pgmap v386: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-24T23:15:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:31 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:15:34.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:33 smithi187 ceph-mon[33292]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:34.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:33 smithi112 ceph-mon[29186]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:34.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:33 smithi150 ceph-mon[32247]: pgmap v387: 129 pgs: 129 active+clean; 656 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:36.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:35 smithi187 ceph-mon[33292]: pgmap v388: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:36.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:35 smithi112 ceph-mon[29186]: pgmap v388: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:35 smithi150 ceph-mon[32247]: pgmap v388: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:38.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:37 smithi187 ceph-mon[33292]: pgmap v389: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:38.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:37 smithi112 ceph-mon[29186]: pgmap v389: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:37 smithi150 ceph-mon[32247]: pgmap v389: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:40.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:39 smithi187 ceph-mon[33292]: pgmap v390: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:40.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:39 smithi112 ceph-mon[29186]: pgmap v390: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:39 smithi150 ceph-mon[32247]: pgmap v390: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:42.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:41 smithi187 ceph-mon[33292]: pgmap v391: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:42.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:41 smithi112 ceph-mon[29186]: pgmap v391: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:41 smithi150 ceph-mon[32247]: pgmap v391: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:44.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:43 smithi187 ceph-mon[33292]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:15:44.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:43 smithi112 ceph-mon[29186]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:15:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:43 smithi150 ceph-mon[32247]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-24T23:15:46.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:45 smithi187 ceph-mon[33292]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:45 smithi112 ceph-mon[29186]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:45 smithi150 ceph-mon[32247]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-04-24T23:15:48.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:47 smithi187 ceph-mon[33292]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:48.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:47 smithi112 ceph-mon[29186]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:48.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:47 smithi150 ceph-mon[32247]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:50.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:49 smithi187 ceph-mon[33292]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:49 smithi112 ceph-mon[29186]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:49 smithi150 ceph-mon[32247]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:51 smithi187 ceph-mon[33292]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:51 smithi112 ceph-mon[29186]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:51 smithi150 ceph-mon[32247]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:53 smithi187 ceph-mon[33292]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:53 smithi112 ceph-mon[29186]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:53 smithi150 ceph-mon[32247]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:15:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:55 smithi187 ceph-mon[33292]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:55 smithi112 ceph-mon[29186]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:55 smithi150 ceph-mon[32247]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:57 smithi187 ceph-mon[33292]: pgmap v399: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:58.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:57 smithi112 ceph-mon[29186]: pgmap v399: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:57 smithi150 ceph-mon[32247]: pgmap v399: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:15:58.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:15:58.499 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:15:58.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/95710824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0104b30 msgr2=0x7fd6f0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/95710824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0104b30 0x7fd6f0106f20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009900 tx=0x7fd6e402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:58.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/95710824 shutdown_connections 2024-04-24T23:15:58.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/95710824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f0107460 0x7fd6f01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/95710824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0104b30 0x7fd6f0106f20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/95710824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6f0102200 0x7fd6f01045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/95710824 >> 172.21.15.112:0/95710824 conn(0x7fd6f00fbbf0 msgr2=0x7fd6f00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/95710824 shutdown_connections 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.600+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/95710824 wait complete. 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6f4d3b640 1 Processor -- start 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6f4d3b640 1 -- start start 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6f4d3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6f0102200 0x7fd6f0101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6f4d3b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f0104b30 0x7fd6f01001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6f4d3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0107460 0x7fd6f01006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6f4d3b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010d580 con 0x7fd6f0107460 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6f4d3b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010d400 con 0x7fd6f0102200 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6f4d3b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f010d700 con 0x7fd6f0104b30 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0107460 0x7fd6f01006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6eed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0107460 0x7fd6f01006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51668/0 (socket says 172.21.15.112:51668) 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6eed9d640 1 -- 172.21.15.112:0/1099954156 learned_addr learned my addr 172.21.15.112:0/1099954156 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f0104b30 0x7fd6f01001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6f0102200 0x7fd6f0101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6eed9d640 1 -- 172.21.15.112:0/1099954156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f0104b30 msgr2=0x7fd6f01001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6eed9d640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f0104b30 0x7fd6f01001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6eed9d640 1 -- 172.21.15.112:0/1099954156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6f0102200 msgr2=0x7fd6f0101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6eed9d640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6f0102200 0x7fd6f0101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6eed9d640 1 -- 172.21.15.112:0/1099954156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f0100f80 con 0x7fd6f0107460 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.601+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f0104b30 0x7fd6f01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.602+0000 7fd6eed9d640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0107460 0x7fd6f01006f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800eb10 tx=0x7fd6e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.602+0000 7fd6df7fe640 1 -- 172.21.15.112:0/1099954156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8004b20 con 0x7fd6f0107460 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.602+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/1099954156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f01a1c80 con 0x7fd6f0107460 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.602+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/1099954156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a21a0 con 0x7fd6f0107460 2024-04-24T23:15:58.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.602+0000 7fd6df7fe640 1 -- 172.21.15.112:0/1099954156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6e8004cc0 con 0x7fd6f0107460 2024-04-24T23:15:58.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.602+0000 7fd6df7fe640 1 -- 172.21.15.112:0/1099954156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80107f0 con 0x7fd6f0107460 2024-04-24T23:15:58.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.605+0000 7fd6f7a47740 1 Processor -- start 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.605+0000 7fd6df7fe640 1 -- 172.21.15.112:0/1099954156 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd6e8005020 con 0x7fd6f0107460 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.605+0000 7fd6f7a47740 1 -- start start 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8a240 0x558065e8a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e8f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 0x558065e92230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558065dee6b0 con 0x558065e8a240 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558065d96880 con 0x558065e8fda0 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558065dea670 con 0x558065e8ac30 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8a240 0x558065e8a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8a240 0x558065e8a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51674/0 (socket says 172.21.15.112:51674) 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6edd9b640 1 -- 172.21.15.112:0/4163860434 learned_addr learned my addr 172.21.15.112:0/4163860434 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6df7fe640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6d80a2c10 0x7fd6d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/4163860434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e8f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6eed9d640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6d80a2c10 0x7fd6d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6edd9b640 1 -- 172.21.15.112:0/4163860434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 msgr2=0x558065e8f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/4163860434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e8f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6edd9b640 1 -- 172.21.15.112:0/4163860434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 msgr2=0x558065e92230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/4163860434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 0x558065e92230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6edd9b640 1 -- 172.21.15.112:0/4163860434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558065bcc7d0 con 0x558065e8a240 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.606+0000 7fd6df7fe640 1 -- 172.21.15.112:0/1099954156 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd6e80cf9f0 con 0x7fd6f0107460 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.607+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/4163860434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8a240 0x558065e8a660 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000c00 tx=0x7fd6e40043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe29ee896d332717 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.607+0000 7fd6dcff9640 1 -- 172.21.15.112:0/4163860434 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4035040 con 0x558065e8a240 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.607+0000 7fd6eed9d640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6d80a2c10 0x7fd6d80a50d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80a45a0 tx=0x7fd6e800efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.607+0000 7fd6dcff9640 1 -- 172.21.15.112:0/4163860434 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6e4004c50 con 0x558065e8a240 2024-04-24T23:15:58.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.607+0000 7fd6dcff9640 1 -- 172.21.15.112:0/4163860434 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4031b10 con 0x558065e8a240 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- 172.21.15.112:0/4163860434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8a240 msgr2=0x558065e8a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/4163860434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8a240 0x558065e8a660 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4000c00 tx=0x7fd6e40043f0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- 172.21.15.112:0/4163860434 shutdown_connections 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/4163860434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 0x558065e92230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/4163860434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e8f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/4163860434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8a240 0x558065e8a660 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- 172.21.15.112:0/4163860434 >> 172.21.15.112:0/4163860434 conn(0x558065e858b0 msgr2=0x558065e85cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- 172.21.15.112:0/4163860434 shutdown_connections 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- 172.21.15.112:0/4163860434 wait complete. 2024-04-24T23:15:58.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- start start 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 Processor -- start 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e94930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8fda0 0x558065e94e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e99ea0 0x558065e9c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558065e92cf0 con 0x558065e8fda0 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558065da9f50 con 0x558065e99ea0 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.608+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558065e92e70 con 0x558065e8ac30 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6f7a47740 10 client.0 _ll_register_callbacks cb 0x558065de5000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e94930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8fda0 0x558065e94e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e94930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33686/0 (socket says 172.21.15.112:33686) 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 -- 172.21.15.112:0/3212769421 learned_addr learned my addr 172.21.15.112:0/3212769421 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e99ea0 0x558065e9c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 -- 172.21.15.112:0/3212769421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e99ea0 msgr2=0x558065e9c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e99ea0 0x558065e9c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 -- 172.21.15.112:0/3212769421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8fda0 msgr2=0x558065e94e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8fda0 0x558065e94e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 -- 172.21.15.112:0/3212769421 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558065b023f0 con 0x558065e8ac30 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8fda0 0x558065e94e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e94930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40316a0 tx=0x7fd6e4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4d786d91063018 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.609+0000 7fd6d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.610+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3212769421 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4004960 con 0x558065e8ac30 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.610+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3212769421 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd6e4031ac0 con 0x558065e8ac30 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.610+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3212769421 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e4004c50 con 0x558065e8ac30 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.610+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3212769421 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6e405a050 con 0x558065e8ac30 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.610+0000 7fd6f7a47740 10 client.25249 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.610+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3212769421 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558065ac20f0 con 0x558065e8ac30 2024-04-24T23:15:58.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.610+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3212769421 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6e4004c50 con 0x558065e8ac30 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.611+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3212769421 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6e4003c40 con 0x558065e8ac30 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.611+0000 7fd6f7a47740 10 client.25249 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.611+0000 7fd6f7a47740 10 client.25249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.611+0000 7fd6f7a47740 20 client.25249 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.611+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x558065ea0780 0x558065ea2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.611+0000 7fd6f7a47740 4 client.25249 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.611+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3212769421 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558065dfbbf0 con 0x558065ea0780 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.611+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x558065ea0780 0x558065ea2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.612+0000 7fd6d5ffb640 10 client.25249 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:58.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.612+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x558065ea0780 0x558065ea2b70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3068fb98b7187889 server_cookie=fcd147dd2c48f3fe in_seq=0 out_seq=0 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3212769421 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558065dfbbf0 con 0x558065ea0780 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6d5ffb640 10 client.25249 handle_command_reply: tid=0 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 client.25249 shutdown 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3212769421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 msgr2=0x558065e94930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e94930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40316a0 tx=0x7fd6e4005850 comp rx=0 tx=0).stop 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3212769421 shutdown_connections 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x558065ea0780 0x558065ea2b70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e99ea0 0x558065e9c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e8fda0 0x558065e94e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3212769421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e94930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3212769421 >> 172.21.15.112:0/3212769421 conn(0x558065e858b0 msgr2=0x558065e84af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6d5ffb640 0 client.25249 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3212769421 shutdown_connections 2024-04-24T23:15:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3212769421 wait complete. 2024-04-24T23:15:58.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.613+0000 7fd6f7a47740 20 client.25249 trim_cache size 0 max 16384 2024-04-24T23:15:58.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.647+0000 7fd6f7a47740 1 Processor -- start 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.647+0000 7fd6f7a47740 1 -- start start 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.647+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e93950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.647+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 0x558065e8b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.647+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e99ea0 0x558065e88910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.647+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01ac870 con 0x558065e99ea0 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.647+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01ae3b0 con 0x558065e8fda0 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.647+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01acf40 con 0x558065e8ac30 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e99ea0 0x558065e88910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e99ea0 0x558065e88910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51700/0 (socket says 172.21.15.112:51700) 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 -- 172.21.15.112:0/437696031 learned_addr learned my addr 172.21.15.112:0/437696031 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e93950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 0x558065e8b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 -- 172.21.15.112:0/437696031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 msgr2=0x558065e93950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:58.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e93950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 -- 172.21.15.112:0/437696031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 msgr2=0x558065e8b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 0x558065e8b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 -- 172.21.15.112:0/437696031 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558065e77090 con 0x558065e99ea0 2024-04-24T23:15:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 0x558065e8b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e99ea0 0x558065e88910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40096c0 tx=0x7fd6e402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eacddd2337cf298 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6d7fff640 1 -- 172.21.15.112:0/437696031 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4000e60 con 0x558065e99ea0 2024-04-24T23:15:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6d7fff640 1 -- 172.21.15.112:0/437696031 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6e40043f0 con 0x558065e99ea0 2024-04-24T23:15:58.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.648+0000 7fd6d7fff640 1 -- 172.21.15.112:0/437696031 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4035070 con 0x558065e99ea0 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 -- 172.21.15.112:0/437696031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e99ea0 msgr2=0x558065e88910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e99ea0 0x558065e88910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40096c0 tx=0x7fd6e402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 -- 172.21.15.112:0/437696031 shutdown_connections 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065e99ea0 0x558065e88910 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8fda0 0x558065e8b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/437696031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8ac30 0x558065e93950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 -- 172.21.15.112:0/437696031 >> 172.21.15.112:0/437696031 conn(0x558065e858b0 msgr2=0x558065e93240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 -- 172.21.15.112:0/437696031 shutdown_connections 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 -- 172.21.15.112:0/437696031 wait complete. 2024-04-24T23:15:58.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.649+0000 7fd6f7a47740 1 -- start start 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6f7a47740 1 Processor -- start 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8ac30 0x558065eb1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8fda0 0x558065eb17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6f7a47740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065eb27c0 0x558065eb2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01adce0 con 0x558065eb27c0 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01ad610 con 0x558065e8ac30 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6f7a47740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01aa850 con 0x558065e8fda0 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6f7a47740 10 client.0 _ll_register_callbacks cb 0x7fd6e000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065eb27c0 0x558065eb2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065eb27c0 0x558065eb2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51704/0 (socket says 172.21.15.112:51704) 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 -- 172.21.15.112:0/3756411986 learned_addr learned my addr 172.21.15.112:0/3756411986 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8fda0 0x558065eb17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8ac30 0x558065eb1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 -- 172.21.15.112:0/3756411986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8fda0 msgr2=0x558065eb17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8fda0 0x558065eb17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 -- 172.21.15.112:0/3756411986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8ac30 msgr2=0x558065eb1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8ac30 0x558065eb1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 -- 172.21.15.112:0/3756411986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558065eb3180 con 0x558065eb27c0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8fda0 0x558065eb17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6edd9b640 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065eb27c0 0x558065eb2c40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40092a0 tx=0x7fd6e4005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e240b9b605c18ad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.650+0000 7fd6d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3756411986 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4035070 con 0x558065eb27c0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3756411986 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd6e4005100 con 0x558065eb27c0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3756411986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e40043f0 con 0x558065eb27c0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3756411986 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd6e4003e60 con 0x558065eb27c0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 10 client.15861 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3756411986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558065f066a0 con 0x558065eb27c0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3756411986 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd6e40043f0 con 0x558065eb27c0 2024-04-24T23:15:58.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3756411986 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd6e403c8e0 con 0x558065eb27c0 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 20 client.15861 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x558065f06ee0 0x558065f09300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 4 client.15861 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3756411986 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558065df5f40 con 0x558065f06ee0 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.651+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x558065f06ee0 0x558065f09300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6d5ffb640 10 client.15861 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6ee59c640 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x558065f06ee0 0x558065f09300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f277d59868f4926e server_cookie=6bff31ee4e52efd1 in_seq=0 out_seq=0 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6d5ffb640 1 -- 172.21.15.112:0/3756411986 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558065df5f40 con 0x558065f06ee0 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6d5ffb640 10 client.15861 handle_command_reply: tid=0 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 client.15861 shutdown 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3756411986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065eb27c0 msgr2=0x558065eb2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065eb27c0 0x558065eb2c40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40092a0 tx=0x7fd6e4005a70 comp rx=0 tx=0).stop 2024-04-24T23:15:58.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3756411986 shutdown_connections 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x558065f06ee0 0x558065f09300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558065eb27c0 0x558065eb2c40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558065e8fda0 0x558065eb17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 --2- 172.21.15.112:0/3756411986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558065e8ac30 0x558065eb1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3756411986 >> 172.21.15.112:0/3756411986 conn(0x558065e858b0 msgr2=0x558065e88910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6d5ffb640 0 client.15861 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3756411986 shutdown_connections 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.652+0000 7fd6f7a47740 1 -- 172.21.15.112:0/3756411986 wait complete. 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f7a47740 20 client.15861 trim_cache size 0 max 16384 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:15:58.660 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-04-24T23:15:58.661 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-04-24T23:15:58.661 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:15:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6d80a2c10 msgr2=0x7fd6d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6d80a2c10 0x7fd6d80a50d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80a45a0 tx=0x7fd6e800efd0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0107460 msgr2=0x7fd6f01006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0107460 0x7fd6f01006f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800eb10 tx=0x7fd6e800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/1099954156 shutdown_connections 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6d80a2c10 0x7fd6d80a50d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6f0107460 0x7fd6f01006f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6f0104b30 0x7fd6f01001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 --2- 172.21.15.112:0/1099954156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6f0102200 0x7fd6f0101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/1099954156 >> 172.21.15.112:0/1099954156 conn(0x7fd6f00fbbf0 msgr2=0x7fd6f0102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/1099954156 shutdown_connections 2024-04-24T23:15:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.653+0000 7fd6f4d3b640 1 -- 172.21.15.112:0/1099954156 wait complete. 2024-04-24T23:15:58.666 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:15:58.667 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:15:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 -- 172.21.15.112:0/826346615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c103350 msgr2=0x7f321c109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 --2- 172.21.15.112:0/826346615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c103350 0x7f321c109be0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3214009ca0 tx=0x7f321402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 -- 172.21.15.112:0/826346615 shutdown_connections 2024-04-24T23:15:58.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 --2- 172.21.15.112:0/826346615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c103350 0x7f321c109be0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 --2- 172.21.15.112:0/826346615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c102990 0x7f321c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 --2- 172.21.15.112:0/826346615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321c1017a0 0x7f321c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 -- 172.21.15.112:0/826346615 >> 172.21.15.112:0/826346615 conn(0x7f321c0fccf0 msgr2=0x7f321c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 -- 172.21.15.112:0/826346615 shutdown_connections 2024-04-24T23:15:58.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.765+0000 7f32233b7640 1 -- 172.21.15.112:0/826346615 wait complete. 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f32233b7640 1 Processor -- start 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f32233b7640 1 -- start start 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f32233b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c1017a0 0x7f321c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f32233b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c102990 0x7f321c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f32233b7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321c103350 0x7f321c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f3221153640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c1017a0 0x7f321c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f32233b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f321c10d550 con 0x7f321c1017a0 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f32233b7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f321c10d3d0 con 0x7f321c103350 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f3221153640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c1017a0 0x7f321c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51714/0 (socket says 172.21.15.112:51714) 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f32233b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f321c10d6d0 con 0x7f321c102990 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f3221153640 1 -- 172.21.15.112:0/1633460757 learned_addr learned my addr 172.21.15.112:0/1633460757 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.766+0000 7f3221954640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321c103350 0x7f321c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f3220952640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c102990 0x7f321c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f3221153640 1 -- 172.21.15.112:0/1633460757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c102990 msgr2=0x7f321c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f3221153640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c102990 0x7f321c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f3221153640 1 -- 172.21.15.112:0/1633460757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321c103350 msgr2=0x7f321c19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f3221153640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321c103350 0x7f321c19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f3221153640 1 -- 172.21.15.112:0/1633460757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c19f770 con 0x7f321c1017a0 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f3221153640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c1017a0 0x7f321c197b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f321800d770 tx=0x7f321800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f32127fc640 1 -- 172.21.15.112:0/1633460757 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218017070 con 0x7f321c1017a0 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f32127fc640 1 -- 172.21.15.112:0/1633460757 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3218009c30 con 0x7f321c1017a0 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c19fa60 con 0x7f321c1017a0 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f32127fc640 1 -- 172.21.15.112:0/1633460757 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218003e40 con 0x7f321c1017a0 2024-04-24T23:15:58.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.767+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c19ffb0 con 0x7f321c1017a0 2024-04-24T23:15:58.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.769+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c108a60 con 0x7f321c1017a0 2024-04-24T23:15:58.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.770+0000 7f32127fc640 1 -- 172.21.15.112:0/1633460757 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3218003fe0 con 0x7f321c1017a0 2024-04-24T23:15:58.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.770+0000 7f32127fc640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32080a2c10 0x7f32080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:58.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.771+0000 7f32127fc640 1 -- 172.21.15.112:0/1633460757 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f32180cedb0 con 0x7f321c1017a0 2024-04-24T23:15:58.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.771+0000 7f3220952640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32080a2c10 0x7f32080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:58.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.771+0000 7f3220952640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32080a2c10 0x7f32080a50d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f321c199060 tx=0x7f320c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:58.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:58.779+0000 7f32127fc640 1 -- 172.21.15.112:0/1633460757 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f32180d3070 con 0x7f321c1017a0 2024-04-24T23:15:59.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.005+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f321c02d170 con 0x7f321c1017a0 2024-04-24T23:15:59.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.005+0000 7f32127fc640 1 -- 172.21.15.112:0/1633460757 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f321809b8c0 con 0x7f321c1017a0 2024-04-24T23:15:59.007 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:15:59.008 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:14:48.128683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24424,24425,24490,24524,24482]},"id":1}]} 2024-04-24T23:15:59.008 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32080a2c10 msgr2=0x7f32080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32080a2c10 0x7f32080a50d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f321c199060 tx=0x7f320c005f90 comp rx=0 tx=0).stop 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c1017a0 msgr2=0x7f321c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c1017a0 0x7f321c197b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f321800d770 tx=0x7f321800dc30 comp rx=0 tx=0).stop 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 shutdown_connections 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f32080a2c10 0x7f32080a50d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f321c103350 0x7f321c19f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f321c102990 0x7f321c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 --2- 172.21.15.112:0/1633460757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f321c1017a0 0x7f321c197b50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 >> 172.21.15.112:0/1633460757 conn(0x7f321c0fccf0 msgr2=0x7f321c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 shutdown_connections 2024-04-24T23:15:59.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.008+0000 7f32233b7640 1 -- 172.21.15.112:0/1633460757 wait complete. 2024-04-24T23:15:59.021 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T23:15:59.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 -- 172.21.15.112:0/3460870462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb00104bd0 msgr2=0x7ffb00106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 --2- 172.21.15.112:0/3460870462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb00104bd0 0x7ffb00106fc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009900 tx=0x7ffae802a410 comp rx=0 tx=0).stop 2024-04-24T23:15:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 -- 172.21.15.112:0/3460870462 shutdown_connections 2024-04-24T23:15:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 --2- 172.21.15.112:0/3460870462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb00107500 0x7ffb001098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 --2- 172.21.15.112:0/3460870462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb00104bd0 0x7ffb00106fc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 --2- 172.21.15.112:0/3460870462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb001022a0 0x7ffb00104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 -- 172.21.15.112:0/3460870462 >> 172.21.15.112:0/3460870462 conn(0x7ffb000fbbf0 msgr2=0x7ffb000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 -- 172.21.15.112:0/3460870462 shutdown_connections 2024-04-24T23:15:59.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.150+0000 7ffb05245640 1 -- 172.21.15.112:0/3460870462 wait complete. 2024-04-24T23:15:59.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.151+0000 7ffb05245640 1 Processor -- start 2024-04-24T23:15:59.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.151+0000 7ffb05245640 1 -- start start 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.151+0000 7ffb05245640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb001022a0 0x7ffb00197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffb05245640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb00107500 0x7ffb00198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffb05245640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0019d380 0x7ffb0019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffb05245640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0010d550 con 0x7ffb0019d380 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffb05245640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0010d3d0 con 0x7ffb00107500 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffb05245640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0010d6d0 con 0x7ffb001022a0 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0019d380 0x7ffb0019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0019d380 0x7ffb0019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51722/0 (socket says 172.21.15.112:51722) 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 -- 172.21.15.112:0/961041099 learned_addr learned my addr 172.21.15.112:0/961041099 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffafed9d640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb001022a0 0x7ffb00197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb00107500 0x7ffb00198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 -- 172.21.15.112:0/961041099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb001022a0 msgr2=0x7ffb00197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb001022a0 0x7ffb00197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 -- 172.21.15.112:0/961041099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb00107500 msgr2=0x7ffb00198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb00107500 0x7ffb00198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 -- 172.21.15.112:0/961041099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0019fde0 con 0x7ffb0019d380 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffafed9d640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb001022a0 0x7ffb00197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffaff59e640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0019d380 0x7ffb0019f770 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800eb10 tx=0x7ffaf800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.152+0000 7ffafcd99640 1 -- 172.21.15.112:0/961041099 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8004b20 con 0x7ffb0019d380 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.153+0000 7ffb05245640 1 -- 172.21.15.112:0/961041099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb001a00d0 con 0x7ffb0019d380 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.153+0000 7ffafcd99640 1 -- 172.21.15.112:0/961041099 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffaf8004cc0 con 0x7ffb0019d380 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.153+0000 7ffb05245640 1 -- 172.21.15.112:0/961041099 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb001a0640 con 0x7ffb0019d380 2024-04-24T23:15:59.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.153+0000 7ffafcd99640 1 -- 172.21.15.112:0/961041099 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80107f0 con 0x7ffb0019d380 2024-04-24T23:15:59.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffafcd99640 1 -- 172.21.15.112:0/961041099 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffaf8005020 con 0x7ffb0019d380 2024-04-24T23:15:59.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffb0802a740 1 Processor -- start 2024-04-24T23:15:59.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffb0802a740 1 -- start start 2024-04-24T23:15:59.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790c230 0x56363790c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x563637911720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563637911d90 0x563637914220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffb0802a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563637870670 con 0x563637911d90 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffb0802a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563637818970 con 0x56363790cc20 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.156+0000 7ffb0802a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56363786c630 con 0x56363790c230 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563637911d90 0x563637914220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafcd99640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffaec0a2d90 0x7ffaec0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563637911d90 0x563637914220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51728/0 (socket says 172.21.15.112:51728) 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaf7fff640 1 -- 172.21.15.112:0/26530544 learned_addr learned my addr 172.21.15.112:0/26530544 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafed9d640 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x563637911720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790c230 0x56363790c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafcd99640 1 -- 172.21.15.112:0/961041099 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ffaf8010590 con 0x7ffb0019d380 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafed9d640 1 -- 172.21.15.112:0/26530544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790c230 msgr2=0x56363790c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafed9d640 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790c230 0x56363790c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafed9d640 1 -- 172.21.15.112:0/26530544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563637911d90 msgr2=0x563637914220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafed9d640 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563637911d90 0x563637914220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaff59e640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffaec0a2d90 0x7ffaec0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafed9d640 1 -- 172.21.15.112:0/26530544 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363783c400 con 0x56363790cc20 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563637911d90 0x563637914220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffafed9d640 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x563637911720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009f50 tx=0x7ffaf0013c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=871d8d4d2ddb9565 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaff59e640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffaec0a2d90 0x7ffaec0a5250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80cced0 tx=0x7ffaf8009a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaf57fa640 1 -- 172.21.15.112:0/26530544 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0004030 con 0x56363790cc20 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaf57fa640 1 -- 172.21.15.112:0/26530544 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffaf00041d0 con 0x56363790cc20 2024-04-24T23:15:59.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.157+0000 7ffaf57fa640 1 -- 172.21.15.112:0/26530544 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0003de0 con 0x56363790cc20 2024-04-24T23:15:59.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 -- 172.21.15.112:0/26530544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 msgr2=0x563637911720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x563637911720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009f50 tx=0x7ffaf0013c60 comp rx=0 tx=0).stop 2024-04-24T23:15:59.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 -- 172.21.15.112:0/26530544 shutdown_connections 2024-04-24T23:15:59.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563637911d90 0x563637914220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x563637911720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 --2- 172.21.15.112:0/26530544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790c230 0x56363790c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 -- 172.21.15.112:0/26530544 >> 172.21.15.112:0/26530544 conn(0x5636379078a0 msgr2=0x563637907ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 -- 172.21.15.112:0/26530544 shutdown_connections 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.158+0000 7ffb0802a740 1 -- 172.21.15.112:0/26530544 wait complete. 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 1 -- start start 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 1 Processor -- start 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x563637916330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x563637916870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 0x56363791d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563637914d40 con 0x56363790c230 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56363782c050 con 0x563637911d90 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x563637916330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563637914ec0 con 0x56363790cc20 2024-04-24T23:15:59.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x563637916330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51738/0 (socket says 172.21.15.112:51738) 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 -- 172.21.15.112:0/1007390809 learned_addr learned my addr 172.21.15.112:0/1007390809 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffb0802a740 10 client.0 _ll_register_callbacks cb 0x563637583be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffafed9d640 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x563637916870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 0x56363791d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 -- 172.21.15.112:0/1007390809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 msgr2=0x563637916870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x563637916870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 -- 172.21.15.112:0/1007390809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 msgr2=0x56363791d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 0x56363791d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.159+0000 7ffaf7fff640 1 -- 172.21.15.112:0/1007390809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636377fddc0 con 0x56363790c230 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x563637916330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffae8031570 tx=0x7ffae8031640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ee5085201b1268 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffae67fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffae67fc640 1 -- 172.21.15.112:0/1007390809 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8035040 con 0x56363790c230 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffae67fc640 1 -- 172.21.15.112:0/1007390809 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ffae8005220 con 0x56363790c230 2024-04-24T23:15:59.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffb0802a740 1 -- 172.21.15.112:0/1007390809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaf00041d0 con 0x56363790c230 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffae67fc640 1 -- 172.21.15.112:0/1007390809 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffae8051050 con 0x56363790c230 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffb0802a740 10 client.15879 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffb0802a740 1 -- 172.21.15.112:0/1007390809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5636375843f0 con 0x56363790c230 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffae67fc640 1 -- 172.21.15.112:0/1007390809 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffaf00041d0 con 0x56363790c230 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffae67fc640 1 -- 172.21.15.112:0/1007390809 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffae8049020 con 0x56363790c230 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffb0802a740 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffb0802a740 10 client.15879 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffb0802a740 20 client.15879 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.160+0000 7ffb0802a740 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5636379221e0 0x563637924650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.161+0000 7ffb0802a740 4 client.15879 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.161+0000 7ffb0802a740 1 -- 172.21.15.112:0/1007390809 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363787dbb0 con 0x5636379221e0 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.161+0000 7ffafed9d640 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5636379221e0 0x563637924650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.161+0000 7ffae67fc640 10 client.15879 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.161+0000 7ffafed9d640 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5636379221e0 0x563637924650 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=69062ab983a6fa18 server_cookie=71cb97db1390b346 in_seq=0 out_seq=0 2024-04-24T23:15:59.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffae67fc640 1 -- 172.21.15.112:0/1007390809 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56363787dbb0 con 0x5636379221e0 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffae67fc640 10 client.15879 handle_command_reply: tid=0 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 client.15879 shutdown 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 -- 172.21.15.112:0/1007390809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 msgr2=0x563637916330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x563637916330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffae8031570 tx=0x7ffae8031640 comp rx=0 tx=0).stop 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 -- 172.21.15.112:0/1007390809 shutdown_connections 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x5636379221e0 0x563637924650 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 0x56363791d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffae67fc640 0 client.15879 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x563637916870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 --2- 172.21.15.112:0/1007390809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x563637916330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 -- 172.21.15.112:0/1007390809 >> 172.21.15.112:0/1007390809 conn(0x5636379078a0 msgr2=0x563637906b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 -- 172.21.15.112:0/1007390809 shutdown_connections 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 1 -- 172.21.15.112:0/1007390809 wait complete. 2024-04-24T23:15:59.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.162+0000 7ffb0802a740 20 client.15879 trim_cache size 0 max 16384 2024-04-24T23:15:59.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffb0802a740 1 Processor -- start 2024-04-24T23:15:59.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffb0802a740 1 -- start start 2024-04-24T23:15:59.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363790a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x56363791d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 0x56363790acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffb0802a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001a89d0 con 0x56363790c230 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffb0802a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001aa510 con 0x563637911d90 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffb0802a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001a90a0 con 0x56363790cc20 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363790a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363790a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51744/0 (socket says 172.21.15.112:51744) 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffaf7fff640 1 -- 172.21.15.112:0/2646313322 learned_addr learned my addr 172.21.15.112:0/2646313322 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.187+0000 7ffafed9d640 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x56363791d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf7fff640 1 -- 172.21.15.112:0/2646313322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 msgr2=0x56363791d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x56363791d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf7fff640 1 -- 172.21.15.112:0/2646313322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 msgr2=0x56363790acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 0x56363790acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf7fff640 1 -- 172.21.15.112:0/2646313322 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56363764e7d0 con 0x56363790c230 2024-04-24T23:15:59.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffafed9d640 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x56363791d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:59.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363790a7b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffae8005a40 tx=0x7ffae802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6244149d3e3c3b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf4ff9640 1 -- 172.21.15.112:0/2646313322 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8041030 con 0x56363790c230 2024-04-24T23:15:59.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf4ff9640 1 -- 172.21.15.112:0/2646313322 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffae804b420 con 0x56363790c230 2024-04-24T23:15:59.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.188+0000 7ffaf4ff9640 1 -- 172.21.15.112:0/2646313322 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8035040 con 0x56363790c230 2024-04-24T23:15:59.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 -- 172.21.15.112:0/2646313322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 msgr2=0x56363790a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363790a7b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffae8005a40 tx=0x7ffae802a410 comp rx=0 tx=0).stop 2024-04-24T23:15:59.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 -- 172.21.15.112:0/2646313322 shutdown_connections 2024-04-24T23:15:59.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563637911d90 0x56363790acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56363790cc20 0x56363791d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 --2- 172.21.15.112:0/2646313322 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363790a7b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 -- 172.21.15.112:0/2646313322 >> 172.21.15.112:0/2646313322 conn(0x5636379078a0 msgr2=0x563637906b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 -- 172.21.15.112:0/2646313322 shutdown_connections 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 -- 172.21.15.112:0/2646313322 wait complete. 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 -- start start 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.189+0000 7ffb0802a740 1 Processor -- start 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363791b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x56363791b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffb0802a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563637911d90 0x56363791c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffb0802a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001a9e40 con 0x56363790c230 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffb0802a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001a9770 con 0x56363790cc20 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffb0802a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffb001a69b0 con 0x563637911d90 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363791b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffb0802a740 10 client.0 _ll_register_callbacks cb 0x7ffae8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363791b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51752/0 (socket says 172.21.15.112:51752) 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 -- 172.21.15.112:0/3269936012 learned_addr learned my addr 172.21.15.112:0/3269936012 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563637911d90 0x56363791c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 -- 172.21.15.112:0/3269936012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563637911d90 msgr2=0x56363791c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563637911d90 0x56363791c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffafed9d640 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x56363791b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 -- 172.21.15.112:0/3269936012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 msgr2=0x56363791b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x56363791b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 -- 172.21.15.112:0/3269936012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56363791cec0 con 0x56363790c230 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffaf7fff640 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363791b380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffae8004650 tx=0x7ffae8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a1ba9e23407c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffae67fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.190+0000 7ffae67fc640 1 -- 172.21.15.112:0/3269936012 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8035070 con 0x56363790c230 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 1 -- 172.21.15.112:0/3269936012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae804b420 con 0x56363790c230 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffae67fc640 1 -- 172.21.15.112:0/3269936012 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ffae804b5d0 con 0x56363790c230 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffae67fc640 1 -- 172.21.15.112:0/3269936012 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffae8041030 con 0x56363790c230 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 10 client.15885 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 1 -- 172.21.15.112:0/3269936012 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563637991360 con 0x56363790c230 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffae67fc640 1 -- 172.21.15.112:0/3269936012 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ffae804b420 con 0x56363790c230 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffae67fc640 1 -- 172.21.15.112:0/3269936012 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffae8049070 con 0x56363790c230 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 10 client.15885 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 10 client.15885 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 20 client.15885 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x563637991c90 0x5636379940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 4 client.15885 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffb0802a740 1 -- 172.21.15.112:0/3269936012 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563637877f00 con 0x563637991c90 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.191+0000 7ffafed9d640 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x563637991c90 0x5636379940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffafed9d640 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x563637991c90 0x5636379940b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6afde0c6d76dda0 server_cookie=b4ef09609ee7de2d in_seq=0 out_seq=0 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffae67fc640 10 client.15885 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffae67fc640 1 -- 172.21.15.112:0/3269936012 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563637877f00 con 0x563637991c90 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffae67fc640 10 client.15885 handle_command_reply: tid=0 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 client.15885 shutdown 2024-04-24T23:15:59.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 -- 172.21.15.112:0/3269936012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 msgr2=0x56363791b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363791b380 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffae8004650 tx=0x7ffae8004040 comp rx=0 tx=0).stop 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 -- 172.21.15.112:0/3269936012 shutdown_connections 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x563637991c90 0x5636379940b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563637911d90 0x56363791c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56363790cc20 0x56363791b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffae67fc640 0 client.15885 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 --2- 172.21.15.112:0/3269936012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56363790c230 0x56363791b380 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 -- 172.21.15.112:0/3269936012 >> 172.21.15.112:0/3269936012 conn(0x5636379078a0 msgr2=0x5636379062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 -- 172.21.15.112:0/3269936012 shutdown_connections 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 1 -- 172.21.15.112:0/3269936012 wait complete. 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.192+0000 7ffb0802a740 20 client.15885 trim_cache size 0 max 16384 2024-04-24T23:15:59.200 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:59.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 -- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffaec0a2d90 msgr2=0x7ffaec0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffaec0a2d90 0x7ffaec0a5250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80cced0 tx=0x7ffaf8009a70 comp rx=0 tx=0).stop 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 -- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0019d380 msgr2=0x7ffb0019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0019d380 0x7ffb0019f770 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800eb10 tx=0x7ffaf800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 -- 172.21.15.112:0/961041099 shutdown_connections 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ffaec0a2d90 0x7ffaec0a5250 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb0019d380 0x7ffb0019f770 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffb00107500 0x7ffb00198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 --2- 172.21.15.112:0/961041099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb001022a0 0x7ffb00197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 -- 172.21.15.112:0/961041099 >> 172.21.15.112:0/961041099 conn(0x7ffb000fbbf0 msgr2=0x7ffb001056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 -- 172.21.15.112:0/961041099 shutdown_connections 2024-04-24T23:15:59.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.193+0000 7ffb05245640 1 -- 172.21.15.112:0/961041099 wait complete. 2024-04-24T23:15:59.206 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.305+0000 7fe48566b640 1 -- 172.21.15.112:0/334189586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480104b30 msgr2=0x7fe480106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.305+0000 7fe48566b640 1 --2- 172.21.15.112:0/334189586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480104b30 0x7fe480106f20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe474009900 tx=0x7fe47402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.305+0000 7fe48566b640 1 -- 172.21.15.112:0/334189586 shutdown_connections 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.305+0000 7fe48566b640 1 --2- 172.21.15.112:0/334189586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe480107460 0x7fe4801098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.305+0000 7fe48566b640 1 --2- 172.21.15.112:0/334189586 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480104b30 0x7fe480106f20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.305+0000 7fe48566b640 1 --2- 172.21.15.112:0/334189586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe480102200 0x7fe4801045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.305+0000 7fe48566b640 1 -- 172.21.15.112:0/334189586 >> 172.21.15.112:0/334189586 conn(0x7fe4800fbbf0 msgr2=0x7fe4800fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.306+0000 7fe48566b640 1 -- 172.21.15.112:0/334189586 shutdown_connections 2024-04-24T23:15:59.308 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.306+0000 7fe48566b640 1 -- 172.21.15.112:0/334189586 wait complete. 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.306+0000 7fe48566b640 1 Processor -- start 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe48566b640 1 -- start start 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe48566b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe480102200 0x7fe480101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe48566b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe480104b30 0x7fe4801001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe48566b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480107460 0x7fe4801006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe48566b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe48010d510 con 0x7fe480107460 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe48566b640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe48010d390 con 0x7fe480102200 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe48566b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe48010d690 con 0x7fe480104b30 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480107460 0x7fe4801006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480107460 0x7fe4801006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51772/0 (socket says 172.21.15.112:51772) 2024-04-24T23:15:59.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 -- 172.21.15.112:0/531935024 learned_addr learned my addr 172.21.15.112:0/531935024 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47effd640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe480102200 0x7fe480101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe480104b30 0x7fe4801001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 -- 172.21.15.112:0/531935024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe480104b30 msgr2=0x7fe4801001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe480104b30 0x7fe4801001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 -- 172.21.15.112:0/531935024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe480102200 msgr2=0x7fe480101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe480102200 0x7fe480101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 -- 172.21.15.112:0/531935024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe480100f80 con 0x7fe480107460 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.307+0000 7fe47f7fe640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480107460 0x7fe4801006f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe47800eb10 tx=0x7fe47800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.308+0000 7fe467fff640 1 -- 172.21.15.112:0/531935024 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478004b20 con 0x7fe480107460 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.308+0000 7fe467fff640 1 -- 172.21.15.112:0/531935024 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe478004cc0 con 0x7fe480107460 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.308+0000 7fe48566b640 1 -- 172.21.15.112:0/531935024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4801a1cb0 con 0x7fe480107460 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.308+0000 7fe467fff640 1 -- 172.21.15.112:0/531935024 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4780107f0 con 0x7fe480107460 2024-04-24T23:15:59.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.308+0000 7fe48566b640 1 -- 172.21.15.112:0/531935024 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4801a21d0 con 0x7fe480107460 2024-04-24T23:15:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.310+0000 7fe467fff640 1 -- 172.21.15.112:0/531935024 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe478005020 con 0x7fe480107460 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.310+0000 7fe48837b740 1 Processor -- start 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.310+0000 7fe48837b740 1 -- start start 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe48837b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0230 0x55c5ed1e0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe48837b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe48837b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe48837b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed144670 con 0x55c5ed1e0230 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe48837b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed0ec970 con 0x55c5ed1e5d90 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe48837b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed140630 con 0x55c5ed1e0c20 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0230 0x55c5ed1e0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0230 0x55c5ed1e0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51778/0 (socket says 172.21.15.112:51778) 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47e7fc640 1 -- 172.21.15.112:0/1458268537 learned_addr learned my addr 172.21.15.112:0/1458268537 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47effd640 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe467fff640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe4680a2c10 0x7fe4680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47effd640 1 -- 172.21.15.112:0/1458268537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 msgr2=0x55c5ed1e8220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47effd640 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1e8220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47effd640 1 -- 172.21.15.112:0/1458268537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0230 msgr2=0x55c5ed1e0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47effd640 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0230 0x55c5ed1e0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47effd640 1 -- 172.21.15.112:0/1458268537 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ed110400 con 0x55c5ed1e0c20 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47f7fe640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe4680a2c10 0x7fe4680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0230 0x55c5ed1e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe467fff640 1 -- 172.21.15.112:0/531935024 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fe4780cfa00 con 0x7fe480107460 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.311+0000 7fe47effd640 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1e5720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe470009c20 tx=0x7fe470011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb593df45282d38a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.312+0000 7fe4657fa640 1 -- 172.21.15.112:0/1458268537 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47001e070 con 0x55c5ed1e0c20 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.312+0000 7fe4657fa640 1 -- 172.21.15.112:0/1458268537 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe470009dd0 con 0x55c5ed1e0c20 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.312+0000 7fe4657fa640 1 -- 172.21.15.112:0/1458268537 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470018910 con 0x55c5ed1e0c20 2024-04-24T23:15:59.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 -- 172.21.15.112:0/1458268537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 msgr2=0x55c5ed1e5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe47f7fe640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe4680a2c10 0x7fe4680a50d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe47809bda0 tx=0x7fe47801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1e5720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe470009c20 tx=0x7fe470011810 comp rx=0 tx=0).stop 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 -- 172.21.15.112:0/1458268537 shutdown_connections 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1e5720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 --2- 172.21.15.112:0/1458268537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0230 0x55c5ed1e0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 -- 172.21.15.112:0/1458268537 >> 172.21.15.112:0/1458268537 conn(0x55c5ed1db8a0 msgr2=0x55c5ed1dbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 -- 172.21.15.112:0/1458268537 shutdown_connections 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 -- 172.21.15.112:0/1458268537 wait complete. 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 -- start start 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 Processor -- start 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.313+0000 7fe48837b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 0x55c5ed1ea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe48837b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe48837b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe48837b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed1e8d40 con 0x55c5ed1e0c20 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe48837b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed100050 con 0x55c5ed1e0230 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe48837b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed1e8ec0 con 0x55c5ed1e5d90 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe48837b740 10 client.0 _ll_register_callbacks cb 0x55c5ece57be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51786/0 (socket says 172.21.15.112:51786) 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 -- 172.21.15.112:0/2080349285 learned_addr learned my addr 172.21.15.112:0/2080349285 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 0x55c5ed1ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 -- 172.21.15.112:0/2080349285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e5d90 msgr2=0x55c5ed1f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 -- 172.21.15.112:0/2080349285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 msgr2=0x55c5ed1ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 0x55c5ed1ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 -- 172.21.15.112:0/2080349285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ed0d1dc0 con 0x55c5ed1e0c20 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe47effd640 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ea950 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe470009bf0 tx=0x7fe4700056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75695132d538fec server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe462ff5640 1 -- 172.21.15.112:0/2080349285 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47001e040 con 0x55c5ed1e0c20 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe462ff5640 1 -- 172.21.15.112:0/2080349285 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fe4700046c0 con 0x55c5ed1e0c20 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.314+0000 7fe48837b740 1 -- 172.21.15.112:0/2080349285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe470009dd0 con 0x55c5ed1e0c20 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe462ff5640 1 -- 172.21.15.112:0/2080349285 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe470032050 con 0x55c5ed1e0c20 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe48837b740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe48837b740 1 -- 172.21.15.112:0/2080349285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5ece583f0 con 0x55c5ed1e0c20 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe462ff5640 1 -- 172.21.15.112:0/2080349285 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe470009dd0 con 0x55c5ed1e0c20 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe462ff5640 1 -- 172.21.15.112:0/2080349285 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe470021030 con 0x55c5ed1e0c20 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe48837b740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe48837b740 10 client.15897 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe48837b740 20 client.15897 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe48837b740 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55c5ed1f5f30 0x55c5ed1f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe48837b740 4 client.15897 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe48837b740 1 -- 172.21.15.112:0/2080349285 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ed151bb0 con 0x55c5ed1f5f30 2024-04-24T23:15:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.315+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55c5ed1f5f30 0x55c5ed1f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.316+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55c5ed1f5f30 0x55c5ed1f83a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=909e6a6b5e422576 server_cookie=d71abb0c5b74e43f in_seq=0 out_seq=0 2024-04-24T23:15:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.316+0000 7fe462ff5640 10 client.15897 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe462ff5640 1 -- 172.21.15.112:0/2080349285 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5ed151bb0 con 0x55c5ed1f5f30 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe462ff5640 10 client.15897 handle_command_reply: tid=0 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 client.15897 shutdown 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 -- 172.21.15.112:0/2080349285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 msgr2=0x55c5ed1ea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ea950 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe470009bf0 tx=0x7fe4700056a0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 -- 172.21.15.112:0/2080349285 shutdown_connections 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55c5ed1f5f30 0x55c5ed1f83a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe462ff5640 0 client.15897 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:15:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ea950 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 --2- 172.21.15.112:0/2080349285 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 0x55c5ed1ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 -- 172.21.15.112:0/2080349285 >> 172.21.15.112:0/2080349285 conn(0x55c5ed1db8a0 msgr2=0x55c5ed1dab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 -- 172.21.15.112:0/2080349285 shutdown_connections 2024-04-24T23:15:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 1 -- 172.21.15.112:0/2080349285 wait complete. 2024-04-24T23:15:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.317+0000 7fe48837b740 20 client.15897 trim_cache size 0 max 16384 2024-04-24T23:15:59.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe48837b740 1 Processor -- start 2024-04-24T23:15:59.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe48837b740 1 -- start start 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe48837b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0230 0x55c5ed1def10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe48837b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe48837b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe48837b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801ac950 con 0x55c5ed1e0c20 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe48837b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801ae490 con 0x55c5ed1e5d90 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe48837b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801ad020 con 0x55c5ed1e0230 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe47effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe47effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1df450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51802/0 (socket says 172.21.15.112:51802) 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.342+0000 7fe47effd640 1 -- 172.21.15.112:0/2608288084 learned_addr learned my addr 172.21.15.112:0/2608288084 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0230 0x55c5ed1def10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47effd640 1 -- 172.21.15.112:0/2608288084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0230 msgr2=0x55c5ed1def10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47effd640 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0230 0x55c5ed1def10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47effd640 1 -- 172.21.15.112:0/2608288084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 msgr2=0x55c5ed1f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47effd640 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47effd640 1 -- 172.21.15.112:0/2608288084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ecf227d0 con 0x55c5ed1e0c20 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe47effd640 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1df450 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe470018ee0 tx=0x7fe470011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d5ac3526e54b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe464ff9640 1 -- 172.21.15.112:0/2608288084 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47001e040 con 0x55c5ed1e0c20 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe464ff9640 1 -- 172.21.15.112:0/2608288084 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe470016020 con 0x55c5ed1e0c20 2024-04-24T23:15:59.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.343+0000 7fe464ff9640 1 -- 172.21.15.112:0/2608288084 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470003e60 con 0x55c5ed1e0c20 2024-04-24T23:15:59.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 -- 172.21.15.112:0/2608288084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 msgr2=0x55c5ed1df450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1df450 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe470018ee0 tx=0x7fe470011e40 comp rx=0 tx=0).stop 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 -- 172.21.15.112:0/2608288084 shutdown_connections 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1df450 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 --2- 172.21.15.112:0/2608288084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0230 0x55c5ed1def10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 -- 172.21.15.112:0/2608288084 >> 172.21.15.112:0/2608288084 conn(0x55c5ed1db8a0 msgr2=0x55c5ed1de800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 -- 172.21.15.112:0/2608288084 shutdown_connections 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.344+0000 7fe48837b740 1 -- 172.21.15.112:0/2608288084 wait complete. 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 1 -- start start 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 1 Processor -- start 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 0x55c5ed1eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801addc0 con 0x55c5ed1e5d90 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801ad6f0 con 0x55c5ed1e0230 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4801aa930 con 0x55c5ed1e0c20 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe48837b740 10 client.0 _ll_register_callbacks cb 0x7fe47001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe47e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe47e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51808/0 (socket says 172.21.15.112:51808) 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe47e7fc640 1 -- 172.21.15.112:0/3290444601 learned_addr learned my addr 172.21.15.112:0/3290444601 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 0x55c5ed1eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.345+0000 7fe47effd640 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe47e7fc640 1 -- 172.21.15.112:0/3290444601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 msgr2=0x55c5ed1ef4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ef4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe47e7fc640 1 -- 172.21.15.112:0/3290444601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 msgr2=0x55c5ed1eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 0x55c5ed1eefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe47e7fc640 1 -- 172.21.15.112:0/3290444601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ed1f0ae0 con 0x55c5ed1e5d90 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe47e7fc640 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f04f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe474009230 tx=0x7fe470005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6d675d33851857 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:59.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe462ff5640 1 -- 172.21.15.112:0/3290444601 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474035040 con 0x55c5ed1e5d90 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe48837b740 1 -- 172.21.15.112:0/3290444601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe470016020 con 0x55c5ed1e5d90 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe462ff5640 1 -- 172.21.15.112:0/3290444601 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fe47404a810 con 0x55c5ed1e5d90 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.346+0000 7fe462ff5640 1 -- 172.21.15.112:0/3290444601 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe474005040 con 0x55c5ed1e5d90 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.348+0000 7fe48837b740 10 client.15903 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.348+0000 7fe48837b740 1 -- 172.21.15.112:0/3290444601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5ed25c4e0 con 0x55c5ed1e5d90 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.348+0000 7fe462ff5640 1 -- 172.21.15.112:0/3290444601 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe470016020 con 0x55c5ed1e5d90 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.348+0000 7fe462ff5640 1 -- 172.21.15.112:0/3290444601 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe47403d450 con 0x55c5ed1e5d90 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.348+0000 7fe48837b740 10 client.15903 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.348+0000 7fe48837b740 10 client.15903 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.348+0000 7fe48837b740 20 client.15903 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.349+0000 7fe48837b740 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55c5ed25ce30 0x55c5ed25f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.349+0000 7fe48837b740 4 client.15903 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.349+0000 7fe48837b740 1 -- 172.21.15.112:0/3290444601 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5ed14bf00 con 0x55c5ed25ce30 2024-04-24T23:15:59.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.349+0000 7fe47effd640 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55c5ed25ce30 0x55c5ed25f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.349+0000 7fe462ff5640 10 client.15903 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:15:59.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.349+0000 7fe47effd640 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55c5ed25ce30 0x55c5ed25f220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f6fb4e09fd97be1 server_cookie=b7638ace72d0947e in_seq=0 out_seq=0 2024-04-24T23:15:59.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe462ff5640 1 -- 172.21.15.112:0/3290444601 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5ed14bf00 con 0x55c5ed25ce30 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe462ff5640 10 client.15903 handle_command_reply: tid=0 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 client.15903 shutdown 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 -- 172.21.15.112:0/3290444601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e5d90 msgr2=0x55c5ed1f04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f04f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe474009230 tx=0x7fe470005350 comp rx=0 tx=0).stop 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 -- 172.21.15.112:0/3290444601 shutdown_connections 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55c5ed25ce30 0x55c5ed25f220 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5ed1e5d90 0x55c5ed1f04f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5ed1e0c20 0x55c5ed1ef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 --2- 172.21.15.112:0/3290444601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5ed1e0230 0x55c5ed1eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 -- 172.21.15.112:0/3290444601 >> 172.21.15.112:0/3290444601 conn(0x55c5ed1db8a0 msgr2=0x55c5ed1df090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe462ff5640 0 client.15903 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 -- 172.21.15.112:0/3290444601 shutdown_connections 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 1 -- 172.21.15.112:0/3290444601 wait complete. 2024-04-24T23:15:59.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48837b740 20 client.15903 trim_cache size 0 max 16384 2024-04-24T23:15:59.356 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:59.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48566b640 1 -- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe4680a2c10 msgr2=0x7fe4680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48566b640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe4680a2c10 0x7fe4680a50d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe47809bda0 tx=0x7fe47801d000 comp rx=0 tx=0).stop 2024-04-24T23:15:59.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48566b640 1 -- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480107460 msgr2=0x7fe4801006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.350+0000 7fe48566b640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480107460 0x7fe4801006f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe47800eb10 tx=0x7fe47800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.351+0000 7fe48566b640 1 -- 172.21.15.112:0/531935024 shutdown_connections 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.351+0000 7fe48566b640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe4680a2c10 0x7fe4680a50d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.351+0000 7fe48566b640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe480107460 0x7fe4801006f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.351+0000 7fe48566b640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe480104b30 0x7fe4801001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.351+0000 7fe48566b640 1 --2- 172.21.15.112:0/531935024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe480102200 0x7fe480101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.351+0000 7fe48566b640 1 -- 172.21.15.112:0/531935024 >> 172.21.15.112:0/531935024 conn(0x7fe4800fbbf0 msgr2=0x7fe480102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.351+0000 7fe48566b640 1 -- 172.21.15.112:0/531935024 shutdown_connections 2024-04-24T23:15:59.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.351+0000 7fe48566b640 1 -- 172.21.15.112:0/531935024 wait complete. 2024-04-24T23:15:59.363 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:15:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 -- 172.21.15.112:0/3414422037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 msgr2=0x7fc2201098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 --2- 172.21.15.112:0/3414422037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 0x7fc2201098b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc218009ca0 tx=0x7fc21802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 -- 172.21.15.112:0/3414422037 shutdown_connections 2024-04-24T23:15:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 --2- 172.21.15.112:0/3414422037 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 0x7fc2201098b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 --2- 172.21.15.112:0/3414422037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc220104b90 0x7fc220106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 --2- 172.21.15.112:0/3414422037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220102260 0x7fc220104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 -- 172.21.15.112:0/3414422037 >> 172.21.15.112:0/3414422037 conn(0x7fc2200fbbf0 msgr2=0x7fc2200fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 -- 172.21.15.112:0/3414422037 shutdown_connections 2024-04-24T23:15:59.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.464+0000 7fc224f77640 1 -- 172.21.15.112:0/3414422037 wait complete. 2024-04-24T23:15:59.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.465+0000 7fc224f77640 1 Processor -- start 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.466+0000 7fc224f77640 1 -- start start 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.466+0000 7fc224f77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc220102260 0x7fc220197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.466+0000 7fc224f77640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220104b90 0x7fc220198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.466+0000 7fc224f77640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 0x7fc22019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.466+0000 7fc224f77640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010d530 con 0x7fc2201074c0 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.466+0000 7fc224f77640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010d3b0 con 0x7fc220104b90 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.466+0000 7fc224f77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010d6b0 con 0x7fc220102260 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 0x7fc22019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220104b90 0x7fc220198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220104b90 0x7fc220198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:53922/0 (socket says 172.21.15.112:53922) 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21dd9b640 1 -- 172.21.15.112:0/140763514 learned_addr learned my addr 172.21.15.112:0/140763514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21ed9d640 1 -- 172.21.15.112:0/140763514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc220102260 msgr2=0x7fc220197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21ed9d640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc220102260 0x7fc220197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21ed9d640 1 -- 172.21.15.112:0/140763514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220104b90 msgr2=0x7fc220198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21ed9d640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220104b90 0x7fc220198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21ed9d640 1 -- 172.21.15.112:0/140763514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22019f710 con 0x7fc2201074c0 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220104b90 0x7fc220198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc21ed9d640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 0x7fc22019f0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc218031a40 tx=0x7fc218031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.467+0000 7fc20f7fe640 1 -- 172.21.15.112:0/140763514 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21802fb60 con 0x7fc2201074c0 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.468+0000 7fc224f77640 1 -- 172.21.15.112:0/140763514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22019f9a0 con 0x7fc2201074c0 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.468+0000 7fc20f7fe640 1 -- 172.21.15.112:0/140763514 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc21802fd00 con 0x7fc2201074c0 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.468+0000 7fc224f77640 1 -- 172.21.15.112:0/140763514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22019ff40 con 0x7fc2201074c0 2024-04-24T23:15:59.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.468+0000 7fc20f7fe640 1 -- 172.21.15.112:0/140763514 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218046950 con 0x7fc2201074c0 2024-04-24T23:15:59.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.471+0000 7fc227d65740 1 Processor -- start 2024-04-24T23:15:59.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.471+0000 7fc20f7fe640 1 -- 172.21.15.112:0/140763514 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc218030ce0 con 0x7fc2201074c0 2024-04-24T23:15:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.471+0000 7fc227d65740 1 -- start start 2024-04-24T23:15:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc227d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e787650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc227d65740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e78c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc227d65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e78f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc227d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56258e6eb670 con 0x56258e787230 2024-04-24T23:15:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc227d65740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56258e693970 con 0x56258e787c20 2024-04-24T23:15:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc227d65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56258e6e7630 con 0x56258e78cd90 2024-04-24T23:15:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e787650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e787650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51830/0 (socket says 172.21.15.112:51830) 2024-04-24T23:15:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 -- 172.21.15.112:0/1838331893 learned_addr learned my addr 172.21.15.112:0/1838331893 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21e59c640 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e78c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e78f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 -- 172.21.15.112:0/1838331893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 msgr2=0x56258e78f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e78f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 -- 172.21.15.112:0/1838331893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 msgr2=0x56258e78c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e78c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21dd9b640 1 -- 172.21.15.112:0/1838331893 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258e6b7400 con 0x56258e787230 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc20f7fe640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2080a2c10 0x7fc2080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.472+0000 7fc21ed9d640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2080a2c10 0x7fc2080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.473+0000 7fc20f7fe640 1 -- 172.21.15.112:0/140763514 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc2180f08e0 con 0x7fc2201074c0 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.473+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e787650 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc214005f80 tx=0x7fc21400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9d453af3e4edc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.473+0000 7fc21ed9d640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2080a2c10 0x7fc2080a50d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b91b0 tx=0x7fc21802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.473+0000 7fc20cff9640 1 -- 172.21.15.112:0/1838331893 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21400e8d0 con 0x56258e787230 2024-04-24T23:15:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.473+0000 7fc20cff9640 1 -- 172.21.15.112:0/1838331893 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc214010460 con 0x56258e787230 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.473+0000 7fc20cff9640 1 -- 172.21.15.112:0/1838331893 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21400f620 con 0x56258e787230 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 -- 172.21.15.112:0/1838331893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 msgr2=0x56258e787650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e787650 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc214005f80 tx=0x7fc21400bcb0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 -- 172.21.15.112:0/1838331893 shutdown_connections 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e78f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e78c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 --2- 172.21.15.112:0/1838331893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e787650 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 -- 172.21.15.112:0/1838331893 >> 172.21.15.112:0/1838331893 conn(0x56258e7828a0 msgr2=0x56258e782ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 -- 172.21.15.112:0/1838331893 shutdown_connections 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.474+0000 7fc227d65740 1 -- 172.21.15.112:0/1838331893 wait complete. 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.475+0000 7fc227d65740 1 -- start start 2024-04-24T23:15:59.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.475+0000 7fc227d65740 1 Processor -- start 2024-04-24T23:15:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.475+0000 7fc227d65740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787230 0x56258e791350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.475+0000 7fc227d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787c20 0x56258e791890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.475+0000 7fc227d65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e7988d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.475+0000 7fc227d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56258e78fce0 con 0x56258e787c20 2024-04-24T23:15:59.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.475+0000 7fc227d65740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56258e6a7050 con 0x56258e787230 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.475+0000 7fc227d65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56258e78fe60 con 0x56258e78cd90 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787230 0x56258e791350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787230 0x56258e791350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:53948/0 (socket says 172.21.15.112:53948) 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 -- 172.21.15.112:0/1145991203 learned_addr learned my addr 172.21.15.112:0/1145991203 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21e59c640 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787c20 0x56258e791890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e7988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc227d65740 10 client.0 _ll_register_callbacks cb 0x56258e3febe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 -- 172.21.15.112:0/1145991203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 msgr2=0x56258e7988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e7988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 -- 172.21.15.112:0/1145991203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787c20 msgr2=0x56258e791890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787c20 0x56258e791890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 -- 172.21.15.112:0/1145991203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258e678dc0 con 0x56258e787230 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21e59c640 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787c20 0x56258e791890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787230 0x56258e791350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc214005780 tx=0x7fc214010d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c063bb8125ca8c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc205ffb640 1 -- 172.21.15.112:0/1145991203 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214027040 con 0x56258e787230 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.476+0000 7fc205ffb640 1 -- 172.21.15.112:0/1145991203 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc21400e8d0 con 0x56258e787230 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc227d65740 1 -- 172.21.15.112:0/1145991203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc214010460 con 0x56258e787230 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc205ffb640 1 -- 172.21.15.112:0/1145991203 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc214034050 con 0x56258e787230 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc227d65740 10 client.25313 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc227d65740 1 -- 172.21.15.112:0/1145991203 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56258e3ff3f0 con 0x56258e787230 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc205ffb640 1 -- 172.21.15.112:0/1145991203 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc214010460 con 0x56258e787230 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc205ffb640 1 -- 172.21.15.112:0/1145991203 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc21401ea60 con 0x56258e787230 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc227d65740 10 client.25313 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc227d65740 10 client.25313 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.477+0000 7fc227d65740 20 client.25313 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.478+0000 7fc227d65740 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x56258e79cf00 0x56258e79f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.478+0000 7fc227d65740 4 client.25313 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.478+0000 7fc227d65740 1 -- 172.21.15.112:0/1145991203 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56258e6f8bb0 con 0x56258e79cf00 2024-04-24T23:15:59.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.478+0000 7fc21e59c640 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x56258e79cf00 0x56258e79f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.478+0000 7fc205ffb640 10 client.25313 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.478+0000 7fc21e59c640 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x56258e79cf00 0x56258e79f330 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=242d349f16ff3b5e server_cookie=cb44e22301a4a81c in_seq=0 out_seq=0 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc205ffb640 1 -- 172.21.15.112:0/1145991203 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56258e6f8bb0 con 0x56258e79cf00 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc205ffb640 10 client.25313 handle_command_reply: tid=0 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 client.25313 shutdown 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 -- 172.21.15.112:0/1145991203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787230 msgr2=0x56258e791350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787230 0x56258e791350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc214005780 tx=0x7fc214010d10 comp rx=0 tx=0).stop 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 -- 172.21.15.112:0/1145991203 shutdown_connections 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x56258e79cf00 0x56258e79f330 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e7988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc205ffb640 0 client.25313 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787c20 0x56258e791890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 --2- 172.21.15.112:0/1145991203 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787230 0x56258e791350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 -- 172.21.15.112:0/1145991203 >> 172.21.15.112:0/1145991203 conn(0x56258e7828a0 msgr2=0x56258e781b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 -- 172.21.15.112:0/1145991203 shutdown_connections 2024-04-24T23:15:59.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 1 -- 172.21.15.112:0/1145991203 wait complete. 2024-04-24T23:15:59.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.479+0000 7fc227d65740 20 client.25313 trim_cache size 0 max 16384 2024-04-24T23:15:59.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc227d65740 1 Processor -- start 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc227d65740 1 -- start start 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc227d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e7917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc227d65740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e798860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc227d65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e7819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc227d65740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201aa5e0 con 0x56258e787230 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc227d65740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201ac120 con 0x56258e787c20 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc227d65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201aacb0 con 0x56258e78cd90 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e7917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e7917f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51838/0 (socket says 172.21.15.112:51838) 2024-04-24T23:15:59.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21dd9b640 1 -- 172.21.15.112:0/3001027048 learned_addr learned my addr 172.21.15.112:0/3001027048 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21e59c640 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e798860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21dd9b640 1 -- 172.21.15.112:0/3001027048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 msgr2=0x56258e7819c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e7819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21dd9b640 1 -- 172.21.15.112:0/3001027048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 msgr2=0x56258e798860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e798860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21dd9b640 1 -- 172.21.15.112:0/3001027048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258e4c97d0 con 0x56258e787230 2024-04-24T23:15:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.493+0000 7fc21e59c640 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e798860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:59.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.494+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e7917f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc214032040 tx=0x7fc2140120d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2debb33ae303c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.494+0000 7fc207fff640 1 -- 172.21.15.112:0/3001027048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214010460 con 0x56258e787230 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.494+0000 7fc207fff640 1 -- 172.21.15.112:0/3001027048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc214045020 con 0x56258e787230 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.494+0000 7fc207fff640 1 -- 172.21.15.112:0/3001027048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214010a80 con 0x56258e787230 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.494+0000 7fc227d65740 1 -- 172.21.15.112:0/3001027048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 msgr2=0x56258e7917f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e7917f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc214032040 tx=0x7fc2140120d0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 -- 172.21.15.112:0/3001027048 shutdown_connections 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e78cd90 0x56258e7819c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e798860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 --2- 172.21.15.112:0/3001027048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e787230 0x56258e7917f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 -- 172.21.15.112:0/3001027048 >> 172.21.15.112:0/3001027048 conn(0x56258e7828a0 msgr2=0x56258e7812b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 -- 172.21.15.112:0/3001027048 shutdown_connections 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 -- 172.21.15.112:0/3001027048 wait complete. 2024-04-24T23:15:59.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 -- start start 2024-04-24T23:15:59.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 Processor -- start 2024-04-24T23:15:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e795c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e78cd90 0x56258e796180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc227d65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e797190 0x56258e797610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.495+0000 7fc21e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e78cd90 0x56258e796180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e795c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21dd9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e795c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:53968/0 (socket says 172.21.15.112:53968) 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e78cd90 0x56258e796180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51842/0 (socket says 172.21.15.112:51842) 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21dd9b640 1 -- 172.21.15.112:0/2691298417 learned_addr learned my addr 172.21.15.112:0/2691298417 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201aba50 con 0x56258e78cd90 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201ab380 con 0x56258e787c20 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201a85c0 con 0x56258e797190 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc227d65740 10 client.0 _ll_register_callbacks cb 0x7fc214012f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e797190 0x56258e797610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21e59c640 1 -- 172.21.15.112:0/2691298417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e797190 msgr2=0x56258e797610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21e59c640 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e797190 0x56258e797610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21e59c640 1 -- 172.21.15.112:0/2691298417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 msgr2=0x56258e795c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21e59c640 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e795c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21e59c640 1 -- 172.21.15.112:0/2691298417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258e797b50 con 0x56258e78cd90 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e797190 0x56258e797610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc21e59c640 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e78cd90 0x56258e796180 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x56258e79fb40 tx=0x7fc2100118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cd3074ad1ced5f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc205ffb640 1 -- 172.21.15.112:0/2691298417 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210019500 con 0x56258e78cd90 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc214045020 con 0x56258e78cd90 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.496+0000 7fc205ffb640 1 -- 172.21.15.112:0/2691298417 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc2100196a0 con 0x56258e78cd90 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc205ffb640 1 -- 172.21.15.112:0/2691298417 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc21002d050 con 0x56258e78cd90 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc227d65740 10 client.15921 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56258e803490 con 0x56258e78cd90 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc205ffb640 1 -- 172.21.15.112:0/2691298417 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc214045020 con 0x56258e78cd90 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc205ffb640 1 -- 172.21.15.112:0/2691298417 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc21000f450 con 0x56258e78cd90 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc227d65740 10 client.15921 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc227d65740 10 client.15921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc227d65740 20 client.15921 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc227d65740 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x56258e803dc0 0x56258e8061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc227d65740 4 client.15921 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56258e6f2f00 con 0x56258e803dc0 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.497+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x56258e803dc0 0x56258e8061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc205ffb640 10 client.15921 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc21dd9b640 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x56258e803dc0 0x56258e8061e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b297321afe61bc1 server_cookie=9985ec7ae8bbb593 in_seq=0 out_seq=0 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc205ffb640 1 -- 172.21.15.112:0/2691298417 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56258e6f2f00 con 0x56258e803dc0 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc205ffb640 10 client.15921 handle_command_reply: tid=0 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 client.15921 shutdown 2024-04-24T23:15:59.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e78cd90 msgr2=0x56258e796180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e78cd90 0x56258e796180 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x56258e79fb40 tx=0x7fc2100118f0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 shutdown_connections 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x56258e803dc0 0x56258e8061e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc205ffb640 0 client.15921 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56258e797190 0x56258e797610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56258e78cd90 0x56258e796180 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 --2- 172.21.15.112:0/2691298417 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56258e787c20 0x56258e795c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 >> 172.21.15.112:0/2691298417 conn(0x56258e7828a0 msgr2=0x56258e781b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 shutdown_connections 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 1 -- 172.21.15.112:0/2691298417 wait complete. 2024-04-24T23:15:59.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.498+0000 7fc227d65740 20 client.15921 trim_cache size 0 max 16384 2024-04-24T23:15:59.507 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:59.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 -- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2080a2c10 msgr2=0x7fc2080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2080a2c10 0x7fc2080a50d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b91b0 tx=0x7fc21802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 -- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 msgr2=0x7fc22019f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 0x7fc22019f0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc218031a40 tx=0x7fc218031ee0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 -- 172.21.15.112:0/140763514 shutdown_connections 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc2080a2c10 0x7fc2080a50d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2201074c0 0x7fc22019f0a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc220104b90 0x7fc220198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 --2- 172.21.15.112:0/140763514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc220102260 0x7fc220197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 -- 172.21.15.112:0/140763514 >> 172.21.15.112:0/140763514 conn(0x7fc2200fbbf0 msgr2=0x7fc220102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 -- 172.21.15.112:0/140763514 shutdown_connections 2024-04-24T23:15:59.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.499+0000 7fc224f77640 1 -- 172.21.15.112:0/140763514 wait complete. 2024-04-24T23:15:59.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:15:59.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 -- 172.21.15.112:0/2991527306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4101350 msgr2=0x7fb2d41017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 --2- 172.21.15.112:0/2991527306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4101350 0x7fb2d41017d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009ca0 tx=0x7fb2cc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 -- 172.21.15.112:0/2991527306 shutdown_connections 2024-04-24T23:15:59.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 --2- 172.21.15.112:0/2991527306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4101350 0x7fb2d41017d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 --2- 172.21.15.112:0/2991527306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d41009b0 0x7fb2d4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 --2- 172.21.15.112:0/2991527306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4100050 0x7fb2d4100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 -- 172.21.15.112:0/2991527306 >> 172.21.15.112:0/2991527306 conn(0x7fb2d40fbbf0 msgr2=0x7fb2d40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 -- 172.21.15.112:0/2991527306 shutdown_connections 2024-04-24T23:15:59.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.619+0000 7fb2d8837640 1 -- 172.21.15.112:0/2991527306 wait complete. 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d8837640 1 Processor -- start 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d8837640 1 -- start start 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d8837640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4100050 0x7fb2d4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d8837640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d41009b0 0x7fb2d419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d8837640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4101350 0x7fb2d41a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d8837640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d410f740 con 0x7fb2d4100050 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4100050 0x7fb2d4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d8837640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d410f5c0 con 0x7fb2d41009b0 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4100050 0x7fb2d4199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51860/0 (socket says 172.21.15.112:51860) 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d259c640 1 -- 172.21.15.112:0/3374319428 learned_addr learned my addr 172.21.15.112:0/3374319428 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d410f8c0 con 0x7fb2d4101350 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d41009b0 0x7fb2d419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.620+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4101350 0x7fb2d41a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2d259c640 1 -- 172.21.15.112:0/3374319428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4101350 msgr2=0x7fb2d41a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4101350 0x7fb2d41a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2d259c640 1 -- 172.21.15.112:0/3374319428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d41009b0 msgr2=0x7fb2d419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d41009b0 0x7fb2d419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2d259c640 1 -- 172.21.15.112:0/3374319428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d41a1940 con 0x7fb2d4100050 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4101350 0x7fb2d41a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4100050 0x7fb2d4199d30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400d950 tx=0x7fb2c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2c37fe640 1 -- 172.21.15.112:0/3374319428 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4017030 con 0x7fb2d4100050 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d41a1c30 con 0x7fb2d4100050 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.621+0000 7fb2c37fe640 1 -- 172.21.15.112:0/3374319428 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb2c4004020 con 0x7fb2d4100050 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.622+0000 7fb2c37fe640 1 -- 172.21.15.112:0/3374319428 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4004a00 con 0x7fb2d4100050 2024-04-24T23:15:59.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.622+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d41a21d0 con 0x7fb2d4100050 2024-04-24T23:15:59.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.624+0000 7fb2c37fe640 1 -- 172.21.15.112:0/3374319428 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb2c40041d0 con 0x7fb2d4100050 2024-04-24T23:15:59.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.625+0000 7fb2db616740 1 Processor -- start 2024-04-24T23:15:59.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.625+0000 7fb2db616740 1 -- start start 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2c37fe640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2bc0a2c10 0x7fb2bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2db616740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aebe650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2db616740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aec3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2db616740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aec6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2db616740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56339ae22670 con 0x56339aec3d90 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2db616740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56339adca970 con 0x56339aebec20 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2db616740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56339ae1e630 con 0x56339aebe230 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d1d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aec3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aec6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d1d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aec3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:54000/0 (socket says 172.21.15.112:54000) 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d1d9b640 1 -- 172.21.15.112:0/653293118 learned_addr learned my addr 172.21.15.112:0/653293118 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d259c640 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aebe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d1d9b640 1 -- 172.21.15.112:0/653293118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 msgr2=0x56339aebe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aebe650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d1d9b640 1 -- 172.21.15.112:0/653293118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 msgr2=0x56339aec6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aec6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d1d9b640 1 -- 172.21.15.112:0/653293118 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56339adee400 con 0x56339aebec20 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aec6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.626+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2bc0a2c10 0x7fb2bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.627+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aec3720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800bc90 tx=0x7fb2c80107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb28c81fd85f460 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.627+0000 7fb2c0ff9640 1 -- 172.21.15.112:0/653293118 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c801c480 con 0x56339aebec20 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.627+0000 7fb2c0ff9640 1 -- 172.21.15.112:0/653293118 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb2c801cb20 con 0x56339aebec20 2024-04-24T23:15:59.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.627+0000 7fb2c0ff9640 1 -- 172.21.15.112:0/653293118 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8011cd0 con 0x56339aebec20 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.627+0000 7fb2c37fe640 1 -- 172.21.15.112:0/3374319428 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb2c40ceed0 con 0x7fb2d4100050 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 -- 172.21.15.112:0/653293118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 msgr2=0x56339aec3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2bc0a2c10 0x7fb2bc0a50d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc004af0 tx=0x7fb2cc02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aec3720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800bc90 tx=0x7fb2c80107a0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 -- 172.21.15.112:0/653293118 shutdown_connections 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aec6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aec3720 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 --2- 172.21.15.112:0/653293118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aebe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 -- 172.21.15.112:0/653293118 >> 172.21.15.112:0/653293118 conn(0x56339aeb98a0 msgr2=0x56339aeb9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 -- 172.21.15.112:0/653293118 shutdown_connections 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 -- 172.21.15.112:0/653293118 wait complete. 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 -- start start 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.628+0000 7fb2db616740 1 Processor -- start 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2db616740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebe230 0x56339aec8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2db616740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aebec20 0x56339aec8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2db616740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aec3d90 0x56339aecf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aebec20 0x56339aec8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aebec20 0x56339aec8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51878/0 (socket says 172.21.15.112:51878) 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d259c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebe230 0x56339aec8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d1d9b640 1 -- 172.21.15.112:0/2439887720 learned_addr learned my addr 172.21.15.112:0/2439887720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56339aec6d40 con 0x56339aebec20 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56339adde050 con 0x56339aebe230 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56339aec6ec0 con 0x56339aec3d90 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aec3d90 0x56339aecf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d259c640 1 -- 172.21.15.112:0/2439887720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aec3d90 msgr2=0x56339aecf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d259c640 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aec3d90 0x56339aecf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d259c640 1 -- 172.21.15.112:0/2439887720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aebec20 msgr2=0x56339aec8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d259c640 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aebec20 0x56339aec8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d259c640 1 -- 172.21.15.112:0/2439887720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56339adafdc0 con 0x56339aebe230 2024-04-24T23:15:59.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aebec20 0x56339aec8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.629+0000 7fb2db616740 10 client.0 _ll_register_callbacks cb 0x56339ab35be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.630+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c801cb20 con 0x56339aebe230 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.630+0000 7fb2d259c640 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebe230 0x56339aec8350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40cc630 tx=0x7fb2c4099350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0be7a99efceaef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.630+0000 7fb2b9ffb640 10 client.25331 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.630+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/2439887720 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c409aca0 con 0x56339aebe230 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.630+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/2439887720 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb2c40d2020 con 0x56339aebe230 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.630+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/2439887720 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2c4099480 con 0x56339aebe230 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.630+0000 7fb2db616740 10 client.25331 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.630+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56339ab363f0 con 0x56339aebe230 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.631+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/2439887720 <== mon.2 v2:172.21.15.150:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb2c409a790 con 0x56339aebe230 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.631+0000 7fb2db616740 10 client.25331 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.631+0000 7fb2db616740 10 client.25331 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.631+0000 7fb2db616740 20 client.25331 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.631+0000 7fb2db616740 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x56339aed3f30 0x56339aed63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.631+0000 7fb2db616740 4 client.25331 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:59.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.631+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56339ae2fbb0 con 0x56339aed3f30 2024-04-24T23:15:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.631+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x56339aed3f30 0x56339aed63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2b9ffb640 10 client.25331 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x56339aed3f30 0x56339aed63f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a40d29b9282a6ff1 server_cookie=9690ce709f81a401 in_seq=0 out_seq=0 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/2439887720 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56339ae2fbb0 con 0x56339aed3f30 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2b9ffb640 10 client.25331 handle_command_reply: tid=0 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 client.25331 shutdown 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebe230 msgr2=0x56339aec8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebe230 0x56339aec8350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40cc630 tx=0x7fb2c4099350 comp rx=0 tx=0).stop 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 shutdown_connections 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x56339aed3f30 0x56339aed63f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2b9ffb640 0 client.25331 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aec3d90 0x56339aecf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aebec20 0x56339aec8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 --2- 172.21.15.112:0/2439887720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebe230 0x56339aec8350 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.632+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 >> 172.21.15.112:0/2439887720 conn(0x56339aeb98a0 msgr2=0x56339aeb8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.633+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 shutdown_connections 2024-04-24T23:15:59.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.633+0000 7fb2db616740 1 -- 172.21.15.112:0/2439887720 wait complete. 2024-04-24T23:15:59.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.633+0000 7fb2db616740 20 client.25331 trim_cache size 0 max 16384 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.657+0000 7fb2db616740 1 Processor -- start 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2db616740 1 -- start start 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2db616740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aec87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2db616740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aecf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2db616740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aeb89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2db616740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d41ac880 con 0x56339aec3d90 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2db616740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d41ae3c0 con 0x56339aebec20 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2db616740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d41acf50 con 0x56339aebe230 2024-04-24T23:15:59.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aeb89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aeb89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51894/0 (socket says 172.21.15.112:51894) 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d2d9d640 1 -- 172.21.15.112:0/3127219774 learned_addr learned my addr 172.21.15.112:0/3127219774 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aec87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aecf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d259c640 1 -- 172.21.15.112:0/3127219774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 msgr2=0x56339aecf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aecf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d259c640 1 -- 172.21.15.112:0/3127219774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 msgr2=0x56339aeb89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aeb89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d259c640 1 -- 172.21.15.112:0/3127219774 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56339ac007d0 con 0x56339aebe230 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.658+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aeb89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.659+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aec87f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4097710 tx=0x7fb2c40d7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61d3085b447b2cb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.659+0000 7fb2bbfff640 1 -- 172.21.15.112:0/3127219774 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c409aca0 con 0x56339aebe230 2024-04-24T23:15:59.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.659+0000 7fb2bbfff640 1 -- 172.21.15.112:0/3127219774 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb2c801cb20 con 0x56339aebe230 2024-04-24T23:15:59.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.659+0000 7fb2bbfff640 1 -- 172.21.15.112:0/3127219774 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40b4330 con 0x56339aebe230 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 -- 172.21.15.112:0/3127219774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 msgr2=0x56339aec87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aec87f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2c4097710 tx=0x7fb2c40d7000 comp rx=0 tx=0).stop 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 -- 172.21.15.112:0/3127219774 shutdown_connections 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aeb89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aecf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 --2- 172.21.15.112:0/3127219774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aec87f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 -- 172.21.15.112:0/3127219774 >> 172.21.15.112:0/3127219774 conn(0x56339aeb98a0 msgr2=0x56339aeb82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 -- 172.21.15.112:0/3127219774 shutdown_connections 2024-04-24T23:15:59.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 -- 172.21.15.112:0/3127219774 wait complete. 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.660+0000 7fb2db616740 1 -- start start 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2db616740 1 Processor -- start 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2db616740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aecd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2db616740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aecd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2db616740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aece5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2db616740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d41adcf0 con 0x56339aec3d90 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2db616740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d41ad620 con 0x56339aebec20 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2db616740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d41aa7f0 con 0x56339aebe230 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aece5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aece5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51902/0 (socket says 172.21.15.112:51902) 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2d2d9d640 1 -- 172.21.15.112:0/3476156993 learned_addr learned my addr 172.21.15.112:0/3476156993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2db616740 10 client.0 _ll_register_callbacks cb 0x7fb2c40051d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aecd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.661+0000 7fb2d1d9b640 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aecd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2d2d9d640 1 -- 172.21.15.112:0/3476156993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 msgr2=0x56339aecd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aecd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2d2d9d640 1 -- 172.21.15.112:0/3476156993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 msgr2=0x56339aecd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aecd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2d2d9d640 1 -- 172.21.15.112:0/3476156993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56339aecebe0 con 0x56339aec3d90 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2d2d9d640 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aece5f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc031ae0 tx=0x7fb2cc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685fdad738b2fa74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/3476156993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc02fcf0 con 0x56339aec3d90 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/3476156993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb2cc0305c0 con 0x56339aec3d90 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2db616740 1 -- 172.21.15.112:0/3476156993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2c801cb20 con 0x56339aec3d90 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/3476156993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2cc05f050 con 0x56339aec3d90 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2db616740 10 client.15939 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.662+0000 7fb2db616740 1 -- 172.21.15.112:0/3476156993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56339af3a6f0 con 0x56339aec3d90 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.663+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/3476156993 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb2cc036040 con 0x56339aec3d90 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.663+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/3476156993 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb2cc049040 con 0x56339aec3d90 2024-04-24T23:15:59.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.663+0000 7fb2db616740 10 client.15939 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.663+0000 7fb2db616740 10 client.15939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.663+0000 7fb2db616740 20 client.15939 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.663+0000 7fb2db616740 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x56339af3b150 0x56339af3d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.663+0000 7fb2db616740 4 client.15939 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.663+0000 7fb2db616740 1 -- 172.21.15.112:0/3476156993 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb2c801cb20 con 0x56339af3b150 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.664+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x56339af3b150 0x56339af3d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.664+0000 7fb2b9ffb640 10 client.15939 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.664+0000 7fb2d259c640 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x56339af3b150 0x56339af3d570 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=510da43668fab4fe server_cookie=1d892be03b1af058 in_seq=0 out_seq=0 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.664+0000 7fb2b9ffb640 1 -- 172.21.15.112:0/3476156993 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb2c801cb20 con 0x56339af3b150 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.664+0000 7fb2b9ffb640 10 client.15939 handle_command_reply: tid=0 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.664+0000 7fb2db616740 1 client.15939 shutdown 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 -- 172.21.15.112:0/3476156993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 msgr2=0x56339aece5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aece5f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc031ae0 tx=0x7fb2cc0095c0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 -- 172.21.15.112:0/3476156993 shutdown_connections 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x56339af3b150 0x56339af3d570 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56339aec3d90 0x56339aece5f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56339aebec20 0x56339aecd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 --2- 172.21.15.112:0/3476156993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56339aebe230 0x56339aecd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 -- 172.21.15.112:0/3476156993 >> 172.21.15.112:0/3476156993 conn(0x56339aeb98a0 msgr2=0x56339aec8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2b9ffb640 0 client.15939 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:15:59.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 -- 172.21.15.112:0/3476156993 shutdown_connections 2024-04-24T23:15:59.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 1 -- 172.21.15.112:0/3476156993 wait complete. 2024-04-24T23:15:59.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2db616740 20 client.15939 trim_cache size 0 max 16384 2024-04-24T23:15:59.672 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:59.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2bc0a2c10 msgr2=0x7fb2bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2d8837640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2bc0a2c10 0x7fb2bc0a50d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc004af0 tx=0x7fb2cc02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4100050 msgr2=0x7fb2d4199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.665+0000 7fb2d8837640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4100050 0x7fb2d4199d30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400d950 tx=0x7fb2c400de10 comp rx=0 tx=0).stop 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.666+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 shutdown_connections 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.666+0000 7fb2d8837640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb2bc0a2c10 0x7fb2bc0a50d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.666+0000 7fb2d8837640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2d4101350 0x7fb2d41a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.666+0000 7fb2d8837640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d41009b0 0x7fb2d419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.666+0000 7fb2d8837640 1 --2- 172.21.15.112:0/3374319428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2d4100050 0x7fb2d4199d30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.666+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 >> 172.21.15.112:0/3374319428 conn(0x7fb2d40fbbf0 msgr2=0x7fb2d40fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.666+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 shutdown_connections 2024-04-24T23:15:59.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.666+0000 7fb2d8837640 1 -- 172.21.15.112:0/3374319428 wait complete. 2024-04-24T23:15:59.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:15:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 -- 172.21.15.112:0/137674215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0101870 msgr2=0x7ff5d0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 --2- 172.21.15.112:0/137674215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0101870 0x7ff5d0101cf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009900 tx=0x7ff5c402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 -- 172.21.15.112:0/137674215 shutdown_connections 2024-04-24T23:15:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 --2- 172.21.15.112:0/137674215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d0102230 0x7ff5d0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 --2- 172.21.15.112:0/137674215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0101870 0x7ff5d0101cf0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 --2- 172.21.15.112:0/137674215 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0100680 0x7ff5d0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 -- 172.21.15.112:0/137674215 >> 172.21.15.112:0/137674215 conn(0x7ff5d00fbbf0 msgr2=0x7ff5d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 -- 172.21.15.112:0/137674215 shutdown_connections 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 -- 172.21.15.112:0/137674215 wait complete. 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.781+0000 7ff5d5722640 1 Processor -- start 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5d5722640 1 -- start start 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5d5722640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d0100680 0x7ff5d0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5d5722640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0101870 0x7ff5d0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5d5722640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102230 0x7ff5d019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5d5722640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d010c430 con 0x7ff5d0102230 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5d5722640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d010c2b0 con 0x7ff5d0101870 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5d5722640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d010c5b0 con 0x7ff5d0100680 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102230 0x7ff5d019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102230 0x7ff5d019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51914/0 (socket says 172.21.15.112:51914) 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/3828051475 learned_addr learned my addr 172.21.15.112:0/3828051475 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d0100680 0x7ff5d0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0101870 0x7ff5d0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/3828051475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d0100680 msgr2=0x7ff5d0197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d0100680 0x7ff5d0197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/3828051475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0101870 msgr2=0x7ff5d0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0101870 0x7ff5d0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/3828051475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d019f7e0 con 0x7ff5d0102230 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.782+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d0100680 0x7ff5d0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.783+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102230 0x7ff5d019f170 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800eb10 tx=0x7ff5c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.783+0000 7ff5b7fff640 1 -- 172.21.15.112:0/3828051475 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8004b20 con 0x7ff5d0102230 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.783+0000 7ff5d5722640 1 -- 172.21.15.112:0/3828051475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d019fad0 con 0x7ff5d0102230 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.783+0000 7ff5b7fff640 1 -- 172.21.15.112:0/3828051475 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff5c8004cc0 con 0x7ff5d0102230 2024-04-24T23:15:59.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.783+0000 7ff5d5722640 1 -- 172.21.15.112:0/3828051475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d01a0040 con 0x7ff5d0102230 2024-04-24T23:15:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.783+0000 7ff5b7fff640 1 -- 172.21.15.112:0/3828051475 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80107f0 con 0x7ff5d0102230 2024-04-24T23:15:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5d8432740 1 Processor -- start 2024-04-24T23:15:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5b7fff640 1 -- 172.21.15.112:0/3828051475 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff5c8010990 con 0x7ff5d0102230 2024-04-24T23:15:59.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5d8432740 1 -- start start 2024-04-24T23:15:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcc91b0 0x55842dcc95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dccc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 0x55842dcd9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5d8432740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55842dc2d670 con 0x55842dcc91b0 2024-04-24T23:15:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5d8432740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55842dbd5970 con 0x55842dcccd00 2024-04-24T23:15:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5d8432740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55842dc29630 con 0x55842dcc9ba0 2024-04-24T23:15:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.786+0000 7ff5ce7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcc91b0 0x55842dcc95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcc91b0 0x55842dcc95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51922/0 (socket says 172.21.15.112:51922) 2024-04-24T23:15:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 -- 172.21.15.112:0/876301212 learned_addr learned my addr 172.21.15.112:0/876301212 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dccc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 0x55842dcd9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 -- 172.21.15.112:0/876301212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 msgr2=0x55842dccc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dccc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 -- 172.21.15.112:0/876301212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 msgr2=0x55842dcd9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 0x55842dcd9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 -- 172.21.15.112:0/876301212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842dbf9400 con 0x55842dcc91b0 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dccc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcc91b0 0x55842dcc95d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005940 tx=0x7ff5c40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146f8098f709cdd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5b7fff640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff5b80a2b90 0x7ff5b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5b7fff640 1 -- 172.21.15.112:0/3828051475 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff5c80cf9a0 con 0x7ff5d0102230 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.787+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff5b80a2b90 0x7ff5b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5b57fa640 1 -- 172.21.15.112:0/876301212 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4035050 con 0x55842dcc91b0 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5b57fa640 1 -- 172.21.15.112:0/876301212 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff5c4031900 con 0x55842dcc91b0 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5b57fa640 1 -- 172.21.15.112:0/876301212 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4005650 con 0x55842dcc91b0 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5d8432740 1 -- 172.21.15.112:0/876301212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcc91b0 msgr2=0x55842dcc95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5d8432740 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcc91b0 0x55842dcc95d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4005940 tx=0x7ff5c40058b0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff5b80a2b90 0x7ff5b80a5050 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55842dcc9fa0 tx=0x7ff5c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5d8432740 1 -- 172.21.15.112:0/876301212 shutdown_connections 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5d8432740 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 0x55842dcd9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5d8432740 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dccc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5d8432740 1 --2- 172.21.15.112:0/876301212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcc91b0 0x55842dcc95d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5d8432740 1 -- 172.21.15.112:0/876301212 >> 172.21.15.112:0/876301212 conn(0x55842dcc48a0 msgr2=0x55842dcc4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.788+0000 7ff5d8432740 1 -- 172.21.15.112:0/876301212 shutdown_connections 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 -- 172.21.15.112:0/876301212 wait complete. 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 -- start start 2024-04-24T23:15:59.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 Processor -- start 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcdbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 0x55842dcdc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dce3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55842dcda560 con 0x55842dce1360 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55842dbe9050 con 0x55842dcccd00 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55842dcda6e0 con 0x55842dcc9ba0 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5d8432740 10 client.0 _ll_register_callbacks cb 0x55842d940be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dce3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5ce7fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 0x55842dcdc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dce3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51924/0 (socket says 172.21.15.112:51924) 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/213730868 learned_addr learned my addr 172.21.15.112:0/213730868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.789+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcdbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/213730868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 msgr2=0x55842dcdbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcdbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/213730868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 msgr2=0x55842dcdc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 0x55842dcdc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/213730868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842dbbadc0 con 0x55842dce1360 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dce3750 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff5c809bda0 tx=0x7ff5c809b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102c54785c0a6693 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/213730868 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c809c270 con 0x55842dce1360 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/213730868 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff5c80a45f0 con 0x55842dce1360 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5d8432740 1 -- 172.21.15.112:0/213730868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c4031900 con 0x55842dce1360 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/213730868 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5c80a01c0 con 0x55842dce1360 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5d8432740 10 client.15951 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5d8432740 1 -- 172.21.15.112:0/213730868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55842d9413f0 con 0x55842dce1360 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.790+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/213730868 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff5c80a4a80 con 0x55842dce1360 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/213730868 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5c809acd0 con 0x55842dce1360 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5d8432740 10 client.15951 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5d8432740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5d8432740 20 client.15951 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5d8432740 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55842dce80a0 0x55842dcea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5d8432740 4 client.15951 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5d8432740 1 -- 172.21.15.112:0/213730868 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff5c4031900 con 0x55842dce80a0 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55842dce80a0 0x55842dcea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.791+0000 7ff5b2ff5640 10 client.15951 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:59.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.792+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55842dce80a0 0x55842dcea4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc5ae2919440413 server_cookie=7564c8fd81fd9270 in_seq=0 out_seq=0 2024-04-24T23:15:59.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.792+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/213730868 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff5c4031900 con 0x55842dce80a0 2024-04-24T23:15:59.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.792+0000 7ff5b2ff5640 10 client.15951 handle_command_reply: tid=0 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.792+0000 7ff5d8432740 1 client.15951 shutdown 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.792+0000 7ff5d8432740 1 -- 172.21.15.112:0/213730868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 msgr2=0x55842dce3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.792+0000 7ff5d8432740 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dce3750 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff5c809bda0 tx=0x7ff5c809b770 comp rx=0 tx=0).stop 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 1 -- 172.21.15.112:0/213730868 shutdown_connections 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55842dce80a0 0x55842dcea4b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5b2ff5640 0 client.15951 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dce3750 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcccd00 0x55842dcdc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 1 --2- 172.21.15.112:0/213730868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcdbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 1 -- 172.21.15.112:0/213730868 >> 172.21.15.112:0/213730868 conn(0x55842dcc48a0 msgr2=0x55842dcc3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 1 -- 172.21.15.112:0/213730868 shutdown_connections 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 1 -- 172.21.15.112:0/213730868 wait complete. 2024-04-24T23:15:59.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.793+0000 7ff5d8432740 20 client.15951 trim_cache size 0 max 16384 2024-04-24T23:15:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5d8432740 1 Processor -- start 2024-04-24T23:15:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5d8432740 1 -- start start 2024-04-24T23:15:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcc9ba0 0x55842dcdb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcccd00 0x55842dcd5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dcd6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5d8432740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01a84a0 con 0x55842dce1360 2024-04-24T23:15:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5d8432740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01a9fe0 con 0x55842dcc9ba0 2024-04-24T23:15:59.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5d8432740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01a8b70 con 0x55842dcccd00 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dcd6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dcd6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51932/0 (socket says 172.21.15.112:51932) 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.806+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/220238287 learned_addr learned my addr 172.21.15.112:0/220238287 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcc9ba0 0x55842dcdb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcccd00 0x55842dcd5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/220238287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcccd00 msgr2=0x55842dcd5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcccd00 0x55842dcd5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/220238287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcc9ba0 msgr2=0x55842dcdb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcc9ba0 0x55842dcdb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5cf7fe640 1 -- 172.21.15.112:0/220238287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55842da0b7d0 con 0x55842dce1360 2024-04-24T23:15:59.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dcd6090 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80049d0 tx=0x7ff5c809a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a786f6477d0c8033 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5b4ff9640 1 -- 172.21.15.112:0/220238287 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80980b0 con 0x55842dce1360 2024-04-24T23:15:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5b4ff9640 1 -- 172.21.15.112:0/220238287 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff5c80a15f0 con 0x55842dce1360 2024-04-24T23:15:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.807+0000 7ff5b4ff9640 1 -- 172.21.15.112:0/220238287 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c809c270 con 0x55842dce1360 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 -- 172.21.15.112:0/220238287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 msgr2=0x55842dcd6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dcd6090 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80049d0 tx=0x7ff5c809a320 comp rx=0 tx=0).stop 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 -- 172.21.15.112:0/220238287 shutdown_connections 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dce1360 0x55842dcd6090 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcccd00 0x55842dcd5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 --2- 172.21.15.112:0/220238287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dcc9ba0 0x55842dcdb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 -- 172.21.15.112:0/220238287 >> 172.21.15.112:0/220238287 conn(0x55842dcc48a0 msgr2=0x55842dcc3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 -- 172.21.15.112:0/220238287 shutdown_connections 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.808+0000 7ff5d8432740 1 -- 172.21.15.112:0/220238287 wait complete. 2024-04-24T23:15:59.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 1 -- start start 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 1 Processor -- start 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcfb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcccd00 0x55842dcf9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dce1360 0x55842dcfa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01a9910 con 0x55842dcccd00 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5ce7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcccd00 0x55842dcf9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01a9240 con 0x55842dce1360 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01a6480 con 0x55842dcc9ba0 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5ce7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcccd00 0x55842dcf9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51946/0 (socket says 172.21.15.112:51946) 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5ce7fc640 1 -- 172.21.15.112:0/16899195 learned_addr learned my addr 172.21.15.112:0/16899195 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5d8432740 10 client.0 _ll_register_callbacks cb 0x7ff5c809bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcfb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.809+0000 7ff5cf7fe640 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dce1360 0x55842dcfa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5ce7fc640 1 -- 172.21.15.112:0/16899195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 msgr2=0x55842dcfb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcfb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5ce7fc640 1 -- 172.21.15.112:0/16899195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dce1360 msgr2=0x55842dcfa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dce1360 0x55842dcfa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5ce7fc640 1 -- 172.21.15.112:0/16899195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55842dcfac90 con 0x55842dcccd00 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcfb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5ce7fc640 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcccd00 0x55842dcf9ee0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009900 tx=0x7ff5c402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826114495d738961 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/16899195 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4035040 con 0x55842dcccd00 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/16899195 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff5c40035f0 con 0x55842dcccd00 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5d8432740 1 -- 172.21.15.112:0/16899195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c80a15f0 con 0x55842dcccd00 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/16899195 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5c4059050 con 0x55842dcccd00 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5d8432740 10 client.15957 fetch_fsmap learned FSMap version 28 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5d8432740 1 -- 172.21.15.112:0/16899195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55842dd4f150 con 0x55842dcccd00 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.810+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/16899195 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff5c4003cf0 con 0x55842dcccd00 2024-04-24T23:15:59.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.811+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/16899195 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff5c403a840 con 0x55842dcccd00 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.811+0000 7ff5d8432740 10 client.15957 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.811+0000 7ff5d8432740 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.811+0000 7ff5d8432740 20 client.15957 populate_metadata read hostname 'smithi112' 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.811+0000 7ff5d8432740 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55842dd4f5a0 0x55842dd51990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.811+0000 7ff5d8432740 4 client.15957 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.811+0000 7ff5d8432740 1 -- 172.21.15.112:0/16899195 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff5c80a15f0 con 0x55842dd4f5a0 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.811+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55842dd4f5a0 0x55842dd51990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5b2ff5640 10 client.15957 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5ceffd640 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55842dd4f5a0 0x55842dd51990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=878978af868107cd server_cookie=c12e6c5818dfb015 in_seq=0 out_seq=0 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5b2ff5640 1 -- 172.21.15.112:0/16899195 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff5c80a15f0 con 0x55842dd4f5a0 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5b2ff5640 10 client.15957 handle_command_reply: tid=0 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 client.15957 shutdown 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 -- 172.21.15.112:0/16899195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcccd00 msgr2=0x55842dcf9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcccd00 0x55842dcf9ee0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009900 tx=0x7ff5c402a410 comp rx=0 tx=0).stop 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 -- 172.21.15.112:0/16899195 shutdown_connections 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55842dd4f5a0 0x55842dd51990 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5b2ff5640 0 client.15957 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55842dce1360 0x55842dcfa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55842dcccd00 0x55842dcf9ee0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 --2- 172.21.15.112:0/16899195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55842dcc9ba0 0x55842dcfb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 -- 172.21.15.112:0/16899195 >> 172.21.15.112:0/16899195 conn(0x55842dcc48a0 msgr2=0x55842dcc32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 -- 172.21.15.112:0/16899195 shutdown_connections 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 1 -- 172.21.15.112:0/16899195 wait complete. 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.812+0000 7ff5d8432740 20 client.15957 trim_cache size 0 max 16384 2024-04-24T23:15:59.820 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:15:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 -- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff5b80a2b90 msgr2=0x7ff5b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff5b80a2b90 0x7ff5b80a5050 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55842dcc9fa0 tx=0x7ff5c000b410 comp rx=0 tx=0).stop 2024-04-24T23:15:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 -- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102230 msgr2=0x7ff5d019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102230 0x7ff5d019f170 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800eb10 tx=0x7ff5c800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 -- 172.21.15.112:0/3828051475 shutdown_connections 2024-04-24T23:15:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff5b80a2b90 0x7ff5b80a5050 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5d0102230 0x7ff5d019f170 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5d0101870 0x7ff5d0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 --2- 172.21.15.112:0/3828051475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5d0100680 0x7ff5d0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 -- 172.21.15.112:0/3828051475 >> 172.21.15.112:0/3828051475 conn(0x7ff5d00fbbf0 msgr2=0x7ff5d00fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 -- 172.21.15.112:0/3828051475 shutdown_connections 2024-04-24T23:15:59.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:15:59.813+0000 7ff5d5722640 1 -- 172.21.15.112:0/3828051475 wait complete. 2024-04-24T23:15:59.961 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:59 smithi112 ceph-mon[29186]: pgmap v400: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:15:59.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:15:59 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1633460757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:59 smithi187 ceph-mon[33292]: pgmap v400: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:16:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:15:59 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1633460757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:59 smithi150 ceph-mon[32247]: pgmap v400: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:16:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:15:59 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1633460757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:00.830 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:16:00.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:16:00.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:16:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 -- 172.21.15.112:0/2950766622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4102260 msgr2=0x7fabc4104650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 --2- 172.21.15.112:0/2950766622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4102260 0x7fabc4104650 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009900 tx=0x7fabb802a410 comp rx=0 tx=0).stop 2024-04-24T23:16:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 -- 172.21.15.112:0/2950766622 shutdown_connections 2024-04-24T23:16:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 --2- 172.21.15.112:0/2950766622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc41074c0 0x7fabc41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 --2- 172.21.15.112:0/2950766622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4104b90 0x7fabc4106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 --2- 172.21.15.112:0/2950766622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc4102260 0x7fabc4104650 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 -- 172.21.15.112:0/2950766622 >> 172.21.15.112:0/2950766622 conn(0x7fabc40fbbf0 msgr2=0x7fabc40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:00.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 -- 172.21.15.112:0/2950766622 shutdown_connections 2024-04-24T23:16:00.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.944+0000 7fabcac9e640 1 -- 172.21.15.112:0/2950766622 wait complete. 2024-04-24T23:16:00.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabcac9e640 1 Processor -- start 2024-04-24T23:16:00.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabcac9e640 1 -- start start 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabcac9e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4104b90 0x7fabc4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabcac9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc41074c0 0x7fabc4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabcac9e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc419d360 0x7fabc419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabcac9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabc410d550 con 0x7fabc41074c0 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabcac9e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabc410d3d0 con 0x7fabc4104b90 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabcac9e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabc410d6d0 con 0x7fabc419d360 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc41074c0 0x7fabc4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc41074c0 0x7fabc4198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46434/0 (socket says 172.21.15.112:46434) 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc3fff640 1 -- 172.21.15.112:0/1251138154 learned_addr learned my addr 172.21.15.112:0/1251138154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4104b90 0x7fabc4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc3fff640 1 -- 172.21.15.112:0/1251138154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc419d360 msgr2=0x7fabc419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc3fff640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc419d360 0x7fabc419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc3fff640 1 -- 172.21.15.112:0/1251138154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4104b90 msgr2=0x7fabc4197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc3fff640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4104b90 0x7fabc4197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc3fff640 1 -- 172.21.15.112:0/1251138154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc419fdc0 con 0x7fabc41074c0 2024-04-24T23:16:00.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.945+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4104b90 0x7fabc4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:00.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.946+0000 7fabc3fff640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc41074c0 0x7fabc4198300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fabb000e9c0 tx=0x7fabb000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:00.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.946+0000 7fabc1ffb640 1 -- 172.21.15.112:0/1251138154 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb000cd80 con 0x7fabc41074c0 2024-04-24T23:16:00.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.946+0000 7fabc1ffb640 1 -- 172.21.15.112:0/1251138154 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fabb0005520 con 0x7fabc41074c0 2024-04-24T23:16:00.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.946+0000 7fabc1ffb640 1 -- 172.21.15.112:0/1251138154 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0010690 con 0x7fabc41074c0 2024-04-24T23:16:00.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.946+0000 7fabcac9e640 1 -- 172.21.15.112:0/1251138154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc41a00b0 con 0x7fabc41074c0 2024-04-24T23:16:00.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.946+0000 7fabcac9e640 1 -- 172.21.15.112:0/1251138154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc41a0650 con 0x7fabc41074c0 2024-04-24T23:16:00.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.949+0000 7fabc1ffb640 1 -- 172.21.15.112:0/1251138154 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fabb00046c0 con 0x7fabc41074c0 2024-04-24T23:16:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.949+0000 7fabcd9a9740 1 Processor -- start 2024-04-24T23:16:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.949+0000 7fabcd9a9740 1 -- start start 2024-04-24T23:16:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.949+0000 7fabc1ffb640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabb40a2c10 0x7fabb40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211d1a0 0x55c44211d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c442122690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c442125190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c44207c2c0 con 0x55c442122d00 2024-04-24T23:16:00.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c442024910 con 0x55c44211d1a0 2024-04-24T23:16:00.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c442078280 con 0x55c44211db90 2024-04-24T23:16:00.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc923b640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabb40a2c10 0x7fabb40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc1ffb640 1 -- 172.21.15.112:0/1251138154 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fabb00cf910 con 0x7fabc41074c0 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc923b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c442125190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc8a3a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211d1a0 0x55c44211d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc8a3a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211d1a0 0x55c44211d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:34560/0 (socket says 172.21.15.112:34560) 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc8a3a640 1 -- 172.21.15.112:0/674152742 learned_addr learned my addr 172.21.15.112:0/674152742 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc3fff640 1 --2- 172.21.15.112:0/674152742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c442122690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc3fff640 1 -- 172.21.15.112:0/674152742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211d1a0 msgr2=0x55c44211d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc3fff640 1 --2- 172.21.15.112:0/674152742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211d1a0 0x55c44211d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc3fff640 1 -- 172.21.15.112:0/674152742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 msgr2=0x55c442125190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc3fff640 1 --2- 172.21.15.112:0/674152742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c442125190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc923b640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabb40a2c10 0x7fabb40a50d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005c40 tx=0x7fabbc012ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.950+0000 7fabc3fff640 1 -- 172.21.15.112:0/674152742 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c442035fb0 con 0x55c44211db90 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.951+0000 7fabc3fff640 1 --2- 172.21.15.112:0/674152742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c442122690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fabb00cd070 tx=0x7fabb009b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=244ba75d3484a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.951+0000 7fabaeffd640 1 -- 172.21.15.112:0/674152742 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0011020 con 0x55c44211db90 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.951+0000 7fabaeffd640 1 -- 172.21.15.112:0/674152742 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fabb009c480 con 0x55c44211db90 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.951+0000 7fabaeffd640 1 -- 172.21.15.112:0/674152742 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0008a80 con 0x55c44211db90 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 -- 172.21.15.112:0/674152742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 msgr2=0x55c442122690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/674152742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c442122690 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fabb00cd070 tx=0x7fabb009b8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 -- 172.21.15.112:0/674152742 shutdown_connections 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/674152742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c442125190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/674152742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c442122690 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/674152742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211d1a0 0x55c44211d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 -- 172.21.15.112:0/674152742 >> 172.21.15.112:0/674152742 conn(0x55c4420e0950 msgr2=0x55c442114810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:00.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 -- 172.21.15.112:0/674152742 shutdown_connections 2024-04-24T23:16:00.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 -- 172.21.15.112:0/674152742 wait complete. 2024-04-24T23:16:00.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 -- start start 2024-04-24T23:16:00.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.952+0000 7fabcd9a9740 1 Processor -- start 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c442127b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c442122d00 0x55c44212eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c442125cb0 con 0x55c44211d1a0 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c442037e90 con 0x55c44211db90 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c442125e30 con 0x55c442122d00 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabcd9a9740 10 client.0 _ll_register_callbacks cb 0x55c442073970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc8a3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c442127b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c442127b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:34574/0 (socket says 172.21.15.112:34574) 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc8a3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46444/0 (socket says 172.21.15.112:46444) 2024-04-24T23:16:00.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 -- 172.21.15.112:0/2253888247 learned_addr learned my addr 172.21.15.112:0/2253888247 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc923b640 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c442122d00 0x55c44212eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 -- 172.21.15.112:0/2253888247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c442122d00 msgr2=0x55c44212eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c442122d00 0x55c44212eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 -- 172.21.15.112:0/2253888247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 msgr2=0x55c4421275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 -- 172.21.15.112:0/2253888247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c441e587e0 con 0x55c44211db90 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7fabc3fff640 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c442127b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabb009a2d0 tx=0x7fabb00112e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd76377442c5909 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.953+0000 7faba3fff640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7faba3fff640 1 -- 172.21.15.112:0/2253888247 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb007f060 con 0x55c44211db90 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 1 -- 172.21.15.112:0/2253888247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb009c480 con 0x55c44211db90 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7faba3fff640 1 -- 172.21.15.112:0/2253888247 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fabb009b1d0 con 0x55c44211db90 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7faba3fff640 1 -- 172.21.15.112:0/2253888247 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fabb00b8070 con 0x55c44211db90 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 10 client.25367 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 1 -- 172.21.15.112:0/2253888247 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c441eb8f70 con 0x55c44211db90 2024-04-24T23:16:00.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7faba3fff640 1 -- 172.21.15.112:0/2253888247 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fabb009c480 con 0x55c44211db90 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7faba3fff640 1 -- 172.21.15.112:0/2253888247 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fabb009ac40 con 0x55c44211db90 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 10 client.25367 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 10 client.25367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 20 client.25367 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c442132ca0 0x55c442135090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 4 client.25367 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.954+0000 7fabcd9a9740 1 -- 172.21.15.112:0/2253888247 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4420505a0 con 0x55c442132ca0 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.955+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c442132ca0 0x55c442135090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.955+0000 7faba3fff640 10 client.25367 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.955+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c442132ca0 0x55c442135090 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9698b0742a710532 server_cookie=999b5d53e7ad5f63 in_seq=0 out_seq=0 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7faba3fff640 1 -- 172.21.15.112:0/2253888247 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4420505a0 con 0x55c442132ca0 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7faba3fff640 10 client.25367 handle_command_reply: tid=0 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 client.25367 shutdown 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 -- 172.21.15.112:0/2253888247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 msgr2=0x55c442127b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c442127b00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabb009a2d0 tx=0x7fabb00112e0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 -- 172.21.15.112:0/2253888247 shutdown_connections 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c442132ca0 0x55c442135090 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c442122d00 0x55c44212eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7faba3fff640 0 client.25367 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c442127b00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/2253888247 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 -- 172.21.15.112:0/2253888247 >> 172.21.15.112:0/2253888247 conn(0x55c4420e0950 msgr2=0x55c442113950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 -- 172.21.15.112:0/2253888247 shutdown_connections 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 1 -- 172.21.15.112:0/2253888247 wait complete. 2024-04-24T23:16:00.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.956+0000 7fabcd9a9740 20 client.25367 trim_cache size 0 max 16384 2024-04-24T23:16:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.974+0000 7fabcd9a9740 1 Processor -- start 2024-04-24T23:16:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.974+0000 7fabcd9a9740 1 -- start start 2024-04-24T23:16:00.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.974+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.974+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c44212eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.974+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c442122d00 0x55c442117910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41a8b30 con 0x55c44211d1a0 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41aa670 con 0x55c442122d00 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41a9200 con 0x55c44211db90 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421173d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46458/0 (socket says 172.21.15.112:46458) 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 -- 172.21.15.112:0/1996101715 learned_addr learned my addr 172.21.15.112:0/1996101715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc3fff640 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c44212eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc923b640 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c442122d00 0x55c442117910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 -- 172.21.15.112:0/1996101715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 msgr2=0x55c44212eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c44212eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 -- 172.21.15.112:0/1996101715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c442122d00 msgr2=0x55c442117910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c442122d00 0x55c442117910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 -- 172.21.15.112:0/1996101715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c442114a60 con 0x55c44211d1a0 2024-04-24T23:16:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc3fff640 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c44212eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc923b640 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c442122d00 0x55c442117910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421173d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009230 tx=0x7fabb80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b475e8de8045cae server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabae7fc640 1 -- 172.21.15.112:0/1996101715 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb804a070 con 0x55c44211d1a0 2024-04-24T23:16:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabae7fc640 1 -- 172.21.15.112:0/1996101715 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fabb8045dc0 con 0x55c44211d1a0 2024-04-24T23:16:00.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.975+0000 7fabae7fc640 1 -- 172.21.15.112:0/1996101715 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8035040 con 0x55c44211d1a0 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 -- 172.21.15.112:0/1996101715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 msgr2=0x55c4421173d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421173d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009230 tx=0x7fabb80058b0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 -- 172.21.15.112:0/1996101715 shutdown_connections 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c442122d00 0x55c442117910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211db90 0x55c44212eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/1996101715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c44211d1a0 0x55c4421173d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 -- 172.21.15.112:0/1996101715 >> 172.21.15.112:0/1996101715 conn(0x55c4420e0950 msgr2=0x55c44211ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 -- 172.21.15.112:0/1996101715 shutdown_connections 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.976+0000 7fabcd9a9740 1 -- 172.21.15.112:0/1996101715 wait complete. 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 1 -- start start 2024-04-24T23:16:00.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 1 Processor -- start 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211d1a0 0x55c44212bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c44212c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c44212d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41a9fa0 con 0x55c442122d00 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41a98d0 con 0x55c44211db90 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabc41a6b10 con 0x55c44211d1a0 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabcd9a9740 10 client.0 _ll_register_callbacks cb 0x7fabb00980d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabc923b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c44212d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabc8a3a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211d1a0 0x55c44212bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabc923b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c44212d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46468/0 (socket says 172.21.15.112:46468) 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabc8a3a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211d1a0 0x55c44212bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:51064/0 (socket says 172.21.15.112:51064) 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabc923b640 1 -- 172.21.15.112:0/397230469 learned_addr learned my addr 172.21.15.112:0/397230469 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.977+0000 7fabc3fff640 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c44212c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabc8a3a640 1 -- 172.21.15.112:0/397230469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 msgr2=0x55c44212c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c44212c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabc8a3a640 1 -- 172.21.15.112:0/397230469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 msgr2=0x55c44212d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c44212d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabc8a3a640 1 -- 172.21.15.112:0/397230469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c441d50170 con 0x55c44211d1a0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabc923b640 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c44212d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabc8a3a640 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211d1a0 0x55c44212bf20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fabb802a450 tx=0x7fabb80056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9974af6711a89e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7faba3fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7faba3fff640 1 -- 172.21.15.112:0/397230469 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8035050 con 0x55c44211d1a0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7faba3fff640 1 -- 172.21.15.112:0/397230469 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fabb8004240 con 0x55c44211d1a0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabcd9a9740 1 -- 172.21.15.112:0/397230469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb8045dc0 con 0x55c44211d1a0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7faba3fff640 1 -- 172.21.15.112:0/397230469 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fabb8004440 con 0x55c44211d1a0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabcd9a9740 10 client.25351 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7fabcd9a9740 1 -- 172.21.15.112:0/397230469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c44212eb40 con 0x55c44211d1a0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.978+0000 7faba3fff640 1 -- 172.21.15.112:0/397230469 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fabb80046c0 con 0x55c44211d1a0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7faba3fff640 1 -- 172.21.15.112:0/397230469 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fabb80035f0 con 0x55c44211d1a0 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7fabcd9a9740 10 client.25351 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7fabcd9a9740 10 client.25351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:00.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7fabcd9a9740 20 client.25351 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c442132ca0 0x55c44211af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7fabcd9a9740 4 client.25351 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7fabcd9a9740 1 -- 172.21.15.112:0/397230469 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fabb8045dc0 con 0x55c442132ca0 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7fabc3fff640 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c442132ca0 0x55c44211af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7fabc3fff640 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c442132ca0 0x55c44211af30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0ba1dc0333c16f server_cookie=95065ab5d443bf04 in_seq=0 out_seq=0 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.979+0000 7faba3fff640 10 client.25351 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.980+0000 7faba3fff640 1 -- 172.21.15.112:0/397230469 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fabb8045dc0 con 0x55c442132ca0 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.980+0000 7faba3fff640 10 client.25351 handle_command_reply: tid=0 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.980+0000 7fabcd9a9740 1 client.25351 shutdown 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 -- 172.21.15.112:0/397230469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211d1a0 msgr2=0x55c44212bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211d1a0 0x55c44212bf20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fabb802a450 tx=0x7fabb80056b0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 -- 172.21.15.112:0/397230469 shutdown_connections 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c442132ca0 0x55c44211af30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c442122d00 0x55c44212d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c44211db90 0x55c44212c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7faba3fff640 0 client.25351 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 --2- 172.21.15.112:0/397230469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c44211d1a0 0x55c44212bf20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 -- 172.21.15.112:0/397230469 >> 172.21.15.112:0/397230469 conn(0x55c4420e0950 msgr2=0x55c4421130c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 -- 172.21.15.112:0/397230469 shutdown_connections 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 1 -- 172.21.15.112:0/397230469 wait complete. 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.981+0000 7fabcd9a9740 20 client.25351 trim_cache size 0 max 16384 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "9f9f2c3e-5430-4c72-8a58-8f9e126b53a7", 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-04-24T23:16:00.987 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:16:00.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 -- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabb40a2c10 msgr2=0x7fabb40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabb40a2c10 0x7fabb40a50d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005c40 tx=0x7fabbc012ab0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 -- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc41074c0 msgr2=0x7fabc4198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:00.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc41074c0 0x7fabc4198300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fabb000e9c0 tx=0x7fabb000ee80 comp rx=0 tx=0).stop 2024-04-24T23:16:00.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 -- 172.21.15.112:0/1251138154 shutdown_connections 2024-04-24T23:16:00.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fabb40a2c10 0x7fabb40a50d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc419d360 0x7fabc419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabc41074c0 0x7fabc4198300 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 --2- 172.21.15.112:0/1251138154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fabc4104b90 0x7fabc4197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:00.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 -- 172.21.15.112:0/1251138154 >> 172.21.15.112:0/1251138154 conn(0x7fabc40fbbf0 msgr2=0x7fabc4102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:00.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 -- 172.21.15.112:0/1251138154 shutdown_connections 2024-04-24T23:16:00.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:00.982+0000 7fabcac9e640 1 -- 172.21.15.112:0/1251138154 wait complete. 2024-04-24T23:16:00.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:16:00.995 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 -- 172.21.15.112:0/2888372578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9781001c0 msgr2=0x7fd978100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 --2- 172.21.15.112:0/2888372578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9781001c0 0x7fd978100670 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd970009ca0 tx=0x7fd97002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 -- 172.21.15.112:0/2888372578 shutdown_connections 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 --2- 172.21.15.112:0/2888372578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9781001c0 0x7fd978100670 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 --2- 172.21.15.112:0/2888372578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9780ff800 0x7fd9780ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 --2- 172.21.15.112:0/2888372578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd978101d10 0x7fd978102130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 -- 172.21.15.112:0/2888372578 >> 172.21.15.112:0/2888372578 conn(0x7fd9780fb380 msgr2=0x7fd9780fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 -- 172.21.15.112:0/2888372578 shutdown_connections 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.141+0000 7fd97ccda640 1 -- 172.21.15.112:0/2888372578 wait complete. 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.142+0000 7fd97ccda640 1 Processor -- start 2024-04-24T23:16:01.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.142+0000 7fd97ccda640 1 -- start start 2024-04-24T23:16:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.142+0000 7fd97ccda640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9780ff800 0x7fd978199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.142+0000 7fd97ccda640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9781001c0 0x7fd97819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.142+0000 7fd97ccda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd978101d10 0x7fd9781a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.142+0000 7fd97ccda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd97810f730 con 0x7fd978101d10 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.142+0000 7fd97ccda640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd97810f5b0 con 0x7fd9780ff800 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.142+0000 7fd97ccda640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd97810f8b0 con 0x7fd9781001c0 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd978101d10 0x7fd9781a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd978101d10 0x7fd9781a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46480/0 (socket says 172.21.15.112:46480) 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 -- 172.21.15.112:0/4096922273 learned_addr learned my addr 172.21.15.112:0/4096922273 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd97659c640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9780ff800 0x7fd978199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd975d9b640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9781001c0 0x7fd97819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 -- 172.21.15.112:0/4096922273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9781001c0 msgr2=0x7fd97819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9781001c0 0x7fd97819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 -- 172.21.15.112:0/4096922273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9780ff800 msgr2=0x7fd978199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9780ff800 0x7fd978199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 -- 172.21.15.112:0/4096922273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9781a1940 con 0x7fd978101d10 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd976d9d640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd978101d10 0x7fd9781a12a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd97002f540 tx=0x7fd970009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd9677fe640 1 -- 172.21.15.112:0/4096922273 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97002fae0 con 0x7fd978101d10 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd97ccda640 1 -- 172.21.15.112:0/4096922273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9781a1bd0 con 0x7fd978101d10 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd9677fe640 1 -- 172.21.15.112:0/4096922273 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd97002fc80 con 0x7fd978101d10 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.143+0000 7fd97ccda640 1 -- 172.21.15.112:0/4096922273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9781a2170 con 0x7fd978101d10 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.144+0000 7fd9677fe640 1 -- 172.21.15.112:0/4096922273 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9700468c0 con 0x7fd978101d10 2024-04-24T23:16:01.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.146+0000 7fd9677fe640 1 -- 172.21.15.112:0/4096922273 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd970046a60 con 0x7fd978101d10 2024-04-24T23:16:01.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.147+0000 7fd9677fe640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9600a2a70 0x7fd9600a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97659c640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9600a2a70 0x7fd9600a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd9677fe640 1 -- 172.21.15.112:0/4096922273 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd9700f0960 con 0x7fd978101d10 2024-04-24T23:16:01.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97f9eb740 1 Processor -- start 2024-04-24T23:16:01.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97f9eb740 1 -- start start 2024-04-24T23:16:01.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28119660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2811e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97659c640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9600a2a70 0x7fd9600a4f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd97819b200 tx=0x7fd96800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c28121230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557c2807d6b0 con 0x557c2811eda0 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c28025880 con 0x557c28119c30 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.148+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557c28079670 con 0x557c28119240 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2811e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd97659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28119660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2811e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:34610/0 (socket says 172.21.15.112:34610) 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd97659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28119660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:51098/0 (socket says 172.21.15.112:51098) 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 -- 172.21.15.112:0/3796976661 learned_addr learned my addr 172.21.15.112:0/3796976661 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd976d9d640 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c28121230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 -- 172.21.15.112:0/3796976661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 msgr2=0x557c28119660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28119660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 -- 172.21.15.112:0/3796976661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 msgr2=0x557c28121230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c28121230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 -- 172.21.15.112:0/3796976661 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c27e5b7d0 con 0x557c28119c30 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd976d9d640 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c28121230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd975d9b640 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2811e730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009c20 tx=0x7fd96c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0244efd3eedb21f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.149+0000 7fd964ff9640 1 -- 172.21.15.112:0/3796976661 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c009e50 con 0x557c28119c30 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd964ff9640 1 -- 172.21.15.112:0/3796976661 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd96c012950 con 0x557c28119c30 2024-04-24T23:16:01.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd964ff9640 1 -- 172.21.15.112:0/3796976661 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00faa0 con 0x557c28119c30 2024-04-24T23:16:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3796976661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 msgr2=0x557c2811e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2811e730 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009c20 tx=0x7fd96c0117b0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3796976661 shutdown_connections 2024-04-24T23:16:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c28121230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2811e730 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/3796976661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28119660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.150+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3796976661 >> 172.21.15.112:0/3796976661 conn(0x557c281148b0 msgr2=0x557c28114cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3796976661 shutdown_connections 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3796976661 wait complete. 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 -- start start 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 Processor -- start 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c28119240 0x557c28123490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119c30 0x557c281239d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c2811eda0 0x557c2812aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557c28121d50 con 0x557c28119240 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557c28038f50 con 0x557c2811eda0 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557c28121ed0 con 0x557c28119c30 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c28119240 0x557c28123490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c28119240 0x557c28123490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46484/0 (socket says 172.21.15.112:46484) 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97659c640 1 -- 172.21.15.112:0/2699567083 learned_addr learned my addr 172.21.15.112:0/2699567083 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:01.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.151+0000 7fd97f9eb740 10 client.0 _ll_register_callbacks cb 0x557c28074000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd975d9b640 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119c30 0x557c281239d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd975d9b640 1 -- 172.21.15.112:0/2699567083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c2811eda0 msgr2=0x557c2812aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd976d9d640 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c2811eda0 0x557c2812aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd975d9b640 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c2811eda0 0x557c2812aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd975d9b640 1 -- 172.21.15.112:0/2699567083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c28119240 msgr2=0x557c28123490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd975d9b640 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c28119240 0x557c28123490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd975d9b640 1 -- 172.21.15.112:0/2699567083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c27d913f0 con 0x557c28119c30 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd97659c640 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c28119240 0x557c28123490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd975d9b640 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119c30 0x557c281239d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x557c28115820 tx=0x7fd96c012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0e44c41f50693b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd95dffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd95dffb640 1 -- 172.21.15.112:0/2699567083 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c028070 con 0x557c28119c30 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd97f9eb740 1 -- 172.21.15.112:0/2699567083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd96c012950 con 0x557c28119c30 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.152+0000 7fd95dffb640 1 -- 172.21.15.112:0/2699567083 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd96c005250 con 0x557c28119c30 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd95dffb640 1 -- 172.21.15.112:0/2699567083 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd96c034050 con 0x557c28119c30 2024-04-24T23:16:01.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97f9eb740 10 client.25363 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97f9eb740 1 -- 172.21.15.112:0/2699567083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c27d510f0 con 0x557c28119c30 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd95dffb640 1 -- 172.21.15.112:0/2699567083 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd96c012950 con 0x557c28119c30 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd95dffb640 1 -- 172.21.15.112:0/2699567083 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd96c0046c0 con 0x557c28119c30 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97f9eb740 10 client.25363 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97f9eb740 10 client.25363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97f9eb740 20 client.25363 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x557c2812f2f0 0x557c28131770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97f9eb740 4 client.25363 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97f9eb740 1 -- 172.21.15.112:0/2699567083 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c2808abf0 con 0x557c2812f2f0 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.153+0000 7fd97659c640 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x557c2812f2f0 0x557c28131770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.154+0000 7fd95dffb640 10 client.25363 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.154+0000 7fd97659c640 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x557c2812f2f0 0x557c28131770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad1953f1ff61706 server_cookie=eafb6ef41678b1b3 in_seq=0 out_seq=0 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.154+0000 7fd95dffb640 1 -- 172.21.15.112:0/2699567083 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c2808abf0 con 0x557c2812f2f0 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.154+0000 7fd95dffb640 10 client.25363 handle_command_reply: tid=0 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 client.25363 shutdown 2024-04-24T23:16:01.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 -- 172.21.15.112:0/2699567083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119c30 msgr2=0x557c281239d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119c30 0x557c281239d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x557c28115820 tx=0x7fd96c012680 comp rx=0 tx=0).stop 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 -- 172.21.15.112:0/2699567083 shutdown_connections 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x557c2812f2f0 0x557c28131770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c2811eda0 0x557c2812aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd95dffb640 0 client.25363 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119c30 0x557c281239d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/2699567083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c28119240 0x557c28123490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 -- 172.21.15.112:0/2699567083 >> 172.21.15.112:0/2699567083 conn(0x557c281148b0 msgr2=0x557c28113af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 -- 172.21.15.112:0/2699567083 shutdown_connections 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 1 -- 172.21.15.112:0/2699567083 wait complete. 2024-04-24T23:16:01.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.155+0000 7fd97f9eb740 20 client.25363 trim_cache size 0 max 16384 2024-04-24T23:16:01.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.189+0000 7fd97f9eb740 1 Processor -- start 2024-04-24T23:16:01.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.189+0000 7fd97f9eb740 1 -- start start 2024-04-24T23:16:01.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.189+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c2811af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.189+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2812a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.189+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c2811b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.189+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781ac600 con 0x557c2811eda0 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.189+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781ae140 con 0x557c28119c30 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.189+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781accd0 con 0x557c28119240 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c2811b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c2811b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46494/0 (socket says 172.21.15.112:46494) 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 -- 172.21.15.112:0/3108774319 learned_addr learned my addr 172.21.15.112:0/3108774319 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd975d9b640 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2812a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 -- 172.21.15.112:0/3108774319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 msgr2=0x557c2811af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd97659c640 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c2811af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c2811af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 -- 172.21.15.112:0/3108774319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 msgr2=0x557c2812a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:01.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2812a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 -- 172.21.15.112:0/3108774319 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2811e210 con 0x557c2811eda0 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd97659c640 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c2811af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd976d9d640 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c2811b490 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd9700bcd60 tx=0x7fd97003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7395609ad9d7a8f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd95ffff640 1 -- 172.21.15.112:0/3108774319 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9700c0070 con 0x557c2811eda0 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd95ffff640 1 -- 172.21.15.112:0/3108774319 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd970036020 con 0x557c2811eda0 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.190+0000 7fd95ffff640 1 -- 172.21.15.112:0/3108774319 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9700b93b0 con 0x557c2811eda0 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.191+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3108774319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 msgr2=0x557c2811b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.191+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c2811b490 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd9700bcd60 tx=0x7fd97003d040 comp rx=0 tx=0).stop 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.191+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3108774319 shutdown_connections 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.191+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c2811b490 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.191+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c2812a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.191+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/3108774319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c2811af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.191+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3108774319 >> 172.21.15.112:0/3108774319 conn(0x557c281148b0 msgr2=0x557c28123490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3108774319 shutdown_connections 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 -- 172.21.15.112:0/3108774319 wait complete. 2024-04-24T23:16:01.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 -- start start 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 Processor -- start 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28127810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c28127d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c281938a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781ada70 con 0x557c2811eda0 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781ad3a0 con 0x557c28119c30 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9781aa5e0 con 0x557c28119240 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd975d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c28127d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd97f9eb740 10 client.0 _ll_register_callbacks cb 0x7fd970031a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd975d9b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c28127d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:34630/0 (socket says 172.21.15.112:34630) 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.192+0000 7fd975d9b640 1 -- 172.21.15.112:0/920045053 learned_addr learned my addr 172.21.15.112:0/920045053 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd976d9d640 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c281938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd97659c640 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28127810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd975d9b640 1 -- 172.21.15.112:0/920045053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 msgr2=0x557c28127810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd975d9b640 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28127810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd975d9b640 1 -- 172.21.15.112:0/920045053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 msgr2=0x557c281938a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd975d9b640 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c281938a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd975d9b640 1 -- 172.21.15.112:0/920045053 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c28193de0 con 0x557c28119c30 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd975d9b640 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c28127d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006010 tx=0x7fd96c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cb178defc2862ba server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd95dffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd95dffb640 1 -- 172.21.15.112:0/920045053 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c028040 con 0x557c28119c30 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd97f9eb740 1 -- 172.21.15.112:0/920045053 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd970036020 con 0x557c28119c30 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.193+0000 7fd95dffb640 1 -- 172.21.15.112:0/920045053 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd96c005250 con 0x557c28119c30 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd95dffb640 1 -- 172.21.15.112:0/920045053 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd96c047050 con 0x557c28119c30 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd97f9eb740 10 client.25385 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd97f9eb740 1 -- 172.21.15.112:0/920045053 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557c28194f80 con 0x557c28119c30 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd95dffb640 1 -- 172.21.15.112:0/920045053 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd970036020 con 0x557c28119c30 2024-04-24T23:16:01.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd95dffb640 1 -- 172.21.15.112:0/920045053 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd96c00fa90 con 0x557c28119c30 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd97f9eb740 10 client.25385 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd97f9eb740 10 client.25385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd97f9eb740 20 client.25385 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x557c28195950 0x557c28197d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd97f9eb740 4 client.25385 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.194+0000 7fd97f9eb740 1 -- 172.21.15.112:0/920045053 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c28084f40 con 0x557c28195950 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97659c640 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x557c28195950 0x557c28197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd95dffb640 10 client.25385 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97659c640 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x557c28195950 0x557c28197d70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74cd80c2ec75044 server_cookie=f398f77bb0df8834 in_seq=0 out_seq=0 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd95dffb640 1 -- 172.21.15.112:0/920045053 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557c28084f40 con 0x557c28195950 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd95dffb640 10 client.25385 handle_command_reply: tid=0 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 client.25385 shutdown 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 -- 172.21.15.112:0/920045053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 msgr2=0x557c28127d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c28127d50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd96c006010 tx=0x7fd96c009f60 comp rx=0 tx=0).stop 2024-04-24T23:16:01.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 -- 172.21.15.112:0/920045053 shutdown_connections 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x557c28195950 0x557c28197d70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd95dffb640 0 client.25385 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557c2811eda0 0x557c281938a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557c28119c30 0x557c28127d50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 --2- 172.21.15.112:0/920045053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557c28119240 0x557c28127810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 -- 172.21.15.112:0/920045053 >> 172.21.15.112:0/920045053 conn(0x557c281148b0 msgr2=0x557c28123470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 -- 172.21.15.112:0/920045053 shutdown_connections 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.195+0000 7fd97f9eb740 1 -- 172.21.15.112:0/920045053 wait complete. 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97f9eb740 20 client.25385 trim_cache size 0 max 16384 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-24T23:16:01.205 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-04-24T23:16:01.206 INFO:teuthology.orchestra.run.smithi112.stdout: "9f9f2c3e-5430-4c72-8a58-8f9e126b53a7": { 2024-04-24T23:16:01.206 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-04-24T23:16:01.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "9f9f2c3e-5430-4c72-8a58-8f9e126b53a7", 2024-04-24T23:16:01.206 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-04-24T23:16:01.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:16:01.206 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:16:01.206 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 -- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9600a2a70 msgr2=0x7fd9600a4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9600a2a70 0x7fd9600a4f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd97819b200 tx=0x7fd96800b410 comp rx=0 tx=0).stop 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 -- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd978101d10 msgr2=0x7fd9781a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd978101d10 0x7fd9781a12a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd97002f540 tx=0x7fd970009890 comp rx=0 tx=0).stop 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 -- 172.21.15.112:0/4096922273 shutdown_connections 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd9600a2a70 0x7fd9600a4f30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd978101d10 0x7fd9781a12a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9781001c0 0x7fd97819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 --2- 172.21.15.112:0/4096922273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9780ff800 0x7fd978199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 -- 172.21.15.112:0/4096922273 >> 172.21.15.112:0/4096922273 conn(0x7fd9780fb380 msgr2=0x7fd9780fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 -- 172.21.15.112:0/4096922273 shutdown_connections 2024-04-24T23:16:01.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:01.196+0000 7fd97ccda640 1 -- 172.21.15.112:0/4096922273 wait complete. 2024-04-24T23:16:01.213 INFO:tasks.cephfs.filesystem:scrub status for tag:9f9f2c3e-5430-4c72-8a58-8f9e126b53a7 - {'path': '/', 'tag': '9f9f2c3e-5430-4c72-8a58-8f9e126b53a7', 'options': 'recursive,force'} 2024-04-24T23:16:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:01 smithi187 ceph-mon[33292]: pgmap v401: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:16:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:01 smithi187 ceph-mon[33292]: scrub queued for path: / 2024-04-24T23:16:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:01 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:01 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:16:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:01 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:01 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:16:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:01 smithi112 ceph-mon[29186]: pgmap v401: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:16:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:01 smithi112 ceph-mon[29186]: scrub queued for path: / 2024-04-24T23:16:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:01 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:01 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:16:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:01 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:01 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:16:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:01 smithi150 ceph-mon[32247]: pgmap v401: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:16:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:01 smithi150 ceph-mon[32247]: scrub queued for path: / 2024-04-24T23:16:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:01 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:01 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:16:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:01 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:01 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:16:04.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:03 smithi187 ceph-mon[33292]: pgmap v402: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:16:04.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:03 smithi112 ceph-mon[29186]: pgmap v402: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:16:04.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:03 smithi150 ceph-mon[32247]: pgmap v402: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:16:06.143 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T23:16:06.143 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-24T23:16:06.143 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:16:06.143 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-24T23:16:06.144 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T23:16:06.147 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T23:16:06.177 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T23:16:06.179 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:16:06.179 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T23:16:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:05 smithi187 ceph-mon[33292]: pgmap v403: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-24T23:16:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:05 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:06.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:05 smithi112 ceph-mon[29186]: pgmap v403: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-24T23:16:06.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:05 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:06.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:05 smithi150 ceph-mon[32247]: pgmap v403: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-24T23:16:06.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:05 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:06.765 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T23:16:06.822 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T23:16:06.858 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:16:06.859 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:16:06.859 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T23:16:06.897 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:16:06.901 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:16:06.901 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T23:16:06.956 INFO:tasks.workunit:Running workunits matching fs/norstats on client.0... 2024-04-24T23:16:06.957 INFO:tasks.workunit:Running workunit fs/norstats/kernel_untar_tar.sh... 2024-04-24T23:16:06.958 DEBUG:teuthology.orchestra.run.smithi112:workunit test fs/norstats/kernel_untar_tar.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/norstats/kernel_untar_tar.sh 2024-04-24T23:16:07.017 INFO:tasks.workunit.client.0.smithi112.stderr:+ KERNEL=linux-4.0.5 2024-04-24T23:16:07.017 INFO:tasks.workunit.client.0.smithi112.stderr:+ wget -q http://download.ceph.com/qa/linux-4.0.5.tar.xz 2024-04-24T23:16:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:07.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:07.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:07.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:07.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:07.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:07.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:16:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:08 smithi112 ceph-mon[29186]: pgmap v404: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-24T23:16:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:08 smithi150 ceph-mon[32247]: pgmap v404: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-24T23:16:08.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:08 smithi187 ceph-mon[33292]: pgmap v404: 129 pgs: 129 active+clean; 661 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-24T23:16:10.143 INFO:tasks.workunit.client.0.smithi112.stderr:+ mkdir untar_tar 2024-04-24T23:16:10.145 INFO:tasks.workunit.client.0.smithi112.stderr:+ cd untar_tar 2024-04-24T23:16:10.146 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar Jxvf ../linux-4.0.5.tar.xz linux-4.0.5/Documentation/ 2024-04-24T23:16:10.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ 2024-04-24T23:16:10.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/00-INDEX 2024-04-24T23:16:10.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/ 2024-04-24T23:16:10.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/README 2024-04-24T23:16:10.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/ 2024-04-24T23:16:10.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/proc-sys-vm-nr_pdflush_threads 2024-04-24T23:16:10.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-bus-usb 2024-04-24T23:16:10.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-class-rfkill 2024-04-24T23:16:10.210 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-koneplus 2024-04-24T23:16:10.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-kovaplus 2024-04-24T23:16:10.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra 2024-04-24T23:16:10.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/ 2024-04-24T23:16:10.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/devfs 2024-04-24T23:16:10.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/dv1394 2024-04-24T23:16:10.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/ip_queue 2024-04-24T23:16:10.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/net_dma 2024-04-24T23:16:10.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/o2cb 2024-04-24T23:16:10.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/raw1394 2024-04-24T23:16:10.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/removed/video1394 2024-04-24T23:16:10.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/ 2024-04-24T23:16:10.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/firewire-cdev 2024-04-24T23:16:10.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/o2cb 2024-04-24T23:16:10.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/syscalls 2024-04-24T23:16:10.228 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-acpi-pmprofile 2024-04-24T23:16:10.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-firewire 2024-04-24T23:16:10.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-usb 2024-04-24T23:16:10.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-bus-xen-backend 2024-04-24T23:16:10.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-backlight 2024-04-24T23:16:10.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-rfkill 2024-04-24T23:16:10.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-tpm 2024-04-24T23:16:10.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-ubi 2024-04-24T23:16:10.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-class-udc 2024-04-24T23:16:10.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-node 2024-04-24T23:16:10.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-cpu 2024-04-24T23:16:10.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-devices-system-xen_memory 2024-04-24T23:16:10.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-ib_srp 2024-04-24T23:16:10.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-qla2xxx 2024-04-24T23:16:10.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-usb-usbtmc 2024-04-24T23:16:10.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-driver-w1_ds28e04 2024-04-24T23:16:10.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-efi-vars 2024-04-24T23:16:10.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-dump 2024-04-24T23:16:10.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-firmware-opal-elog 2024-04-24T23:16:10.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-module 2024-04-24T23:16:10.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/sysfs-transport-srp 2024-04-24T23:16:10.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/thermal-notification 2024-04-24T23:16:10.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/stable/vdso 2024-04-24T23:16:10.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/ 2024-04-24T23:16:10.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-spear-pcie-gadget 2024-04-24T23:16:10.256 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget 2024-04-24T23:16:10.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-acm 2024-04-24T23:16:10.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ecm 2024-04-24T23:16:10.260 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-eem 2024-04-24T23:16:10.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ffs 2024-04-24T23:16:10.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-hid 2024-04-24T23:16:10.263 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-loopback 2024-04-24T23:16:10.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-mass-storage 2024-04-24T23:16:10.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-midi 2024-04-24T23:16:10.266 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-ncm 2024-04-24T23:16:10.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-obex 2024-04-24T23:16:10.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-phonet 2024-04-24T23:16:10.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-rndis 2024-04-24T23:16:10.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-serial 2024-04-24T23:16:10.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-sourcesink 2024-04-24T23:16:10.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-subset 2024-04-24T23:16:10.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac1 2024-04-24T23:16:10.275 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uac2 2024-04-24T23:16:10.276 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/configfs-usb-gadget-uvc 2024-04-24T23:16:10.277 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-driver-genwqe 2024-04-24T23:16:10.278 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ec 2024-04-24T23:16:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:10 smithi112 ceph-mon[29186]: pgmap v405: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-24T23:16:10.279 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-ideapad 2024-04-24T23:16:10.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-olpc 2024-04-24T23:16:10.281 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pfo-nx-crypto 2024-04-24T23:16:10.282 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/debugfs-pktcdvd 2024-04-24T23:16:10.283 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/dev-kmsg 2024-04-24T23:16:10.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/evm 2024-04-24T23:16:10.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/ima_policy 2024-04-24T23:16:10.286 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/procfs-diskstats 2024-04-24T23:16:10.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/pstore 2024-04-24T23:16:10.288 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:10 smithi187 ceph-mon[33292]: pgmap v405: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-24T23:16:10.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ata 2024-04-24T23:16:10.290 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block 2024-04-24T23:16:10.293 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-bcache 2024-04-24T23:16:10.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-dm 2024-04-24T23:16:10.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-rssd 2024-04-24T23:16:10.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-block-zram 2024-04-24T23:16:10.295 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-acpi 2024-04-24T23:16:10.296 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-amba 2024-04-24T23:16:10.297 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-bcma 2024-04-24T23:16:10.298 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etb10 2024-04-24T23:16:10.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-etm3x 2024-04-24T23:16:10.300 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-funnel 2024-04-24T23:16:10.301 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc 2024-04-24T23:16:10.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-css 2024-04-24T23:16:10.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-events 2024-04-24T23:16:10.305 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-format 2024-04-24T23:16:10.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_24x7 2024-04-24T23:16:10.307 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-event_source-devices-hv_gpci 2024-04-24T23:16:10.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-fcoe 2024-04-24T23:16:10.309 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-hsi 2024-04-24T23:16:10.310 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-fsa9480 2024-04-24T23:16:10.311 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 2024-04-24T23:16:10.312 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-i2c-devices-lm3533 2024-04-24T23:16:10.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio 2024-04-24T23:16:10.314 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-accel-bmc150 2024-04-24T23:16:10.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-ad9523 2024-04-24T23:16:10.316 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-frequency-adf4350 2024-04-24T23:16:10.317 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-gyro-bmg160 2024-04-24T23:16:10.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:10 smithi150 ceph-mon[32247]: pgmap v405: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-24T23:16:10.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-light-lm3533-als 2024-04-24T23:16:10.319 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-mpu6050 2024-04-24T23:16:10.320 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 2024-04-24T23:16:10.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-iio-trigger-sysfs 2024-04-24T23:16:10.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mdio 2024-04-24T23:16:10.323 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-media 2024-04-24T23:16:10.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-mei 2024-04-24T23:16:10.325 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci 2024-04-24T23:16:10.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss 2024-04-24T23:16:10.327 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-pci-drivers-ehci_hcd 2024-04-24T23:16:10.328 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-platform 2024-04-24T23:16:10.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rbd 2024-04-24T23:16:10.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-rpmsg 2024-04-24T23:16:10.331 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-umc 2024-04-24T23:16:10.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb 2024-04-24T23:16:10.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg 2024-04-24T23:16:10.334 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-bus-usb-lvstest 2024-04-24T23:16:10.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-c2port 2024-04-24T23:16:10.336 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-cfq-target-latency 2024-04-24T23:16:10.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class 2024-04-24T23:16:10.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-adp8870 2024-04-24T23:16:10.339 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 2024-04-24T23:16:10.341 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-bdi 2024-04-24T23:16:10.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-cxl 2024-04-24T23:16:10.343 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-devfreq 2024-04-24T23:16:10.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-extcon 2024-04-24T23:16:10.345 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu 2024-04-24T23:16:10.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-amd-iommu 2024-04-24T23:16:10.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-iommu-intel-iommu 2024-04-24T23:16:10.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-lcd 2024-04-24T23:16:10.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led 2024-04-24T23:16:10.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 2024-04-24T23:16:10.352 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-leds-gt683r 2024-04-24T23:16:10.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mei 2024-04-24T23:16:10.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mic.txt 2024-04-24T23:16:10.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-mtd 2024-04-24T23:16:10.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net 2024-04-24T23:16:10.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-batman-adv 2024-04-24T23:16:10.358 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-cdc_ncm 2024-04-24T23:16:10.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-grcan 2024-04-24T23:16:10.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-mesh 2024-04-24T23:16:10.361 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-queues 2024-04-24T23:16:10.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-net-statistics 2024-04-24T23:16:10.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pktcdvd 2024-04-24T23:16:10.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-power 2024-04-24T23:16:10.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-powercap 2024-04-24T23:16:10.366 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-pwm 2024-04-24T23:16:10.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rc 2024-04-24T23:16:10.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-regulator 2024-04-24T23:16:10.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-rtc-rtc0-device-rtc_calibration 2024-04-24T23:16:10.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-scsi_host 2024-04-24T23:16:10.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc 2024-04-24T23:16:10.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-class-uwb_rc-wusbhc 2024-04-24T23:16:10.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-dev 2024-04-24T23:16:10.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices 2024-04-24T23:16:10.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-edac 2024-04-24T23:16:10.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-firmware_node 2024-04-24T23:16:10.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-lpss_ltr 2024-04-24T23:16:10.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-memory 2024-04-24T23:16:10.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-mmc 2024-04-24T23:16:10.381 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-online 2024-04-24T23:16:10.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-_UDC_-gadget 2024-04-24T23:16:10.383 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-docg3 2024-04-24T23:16:10.384 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-platform-sh_mobile_lcdc_fb 2024-04-24T23:16:10.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power 2024-04-24T23:16:10.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D0 2024-04-24T23:16:10.387 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D1 2024-04-24T23:16:10.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D2 2024-04-24T23:16:10.389 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_D3hot 2024-04-24T23:16:10.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_resources_wakeup 2024-04-24T23:16:10.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-power_state 2024-04-24T23:16:10.392 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-real_power_state 2024-04-24T23:16:10.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-resource_in_use 2024-04-24T23:16:10.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-soc 2024-04-24T23:16:10.395 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-sun 2024-04-24T23:16:10.396 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-cpu 2024-04-24T23:16:10.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-ibm-rtl 2024-04-24T23:16:10.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-devices-system-xen_cpu 2024-04-24T23:16:10.399 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-genwqe 2024-04-24T23:16:10.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid 2024-04-24T23:16:10.401 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-lenovo 2024-04-24T23:16:10.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-logitech-lg4ff 2024-04-24T23:16:10.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-multitouch 2024-04-24T23:16:10.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-picolcd 2024-04-24T23:16:10.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-prodikeys 2024-04-24T23:16:10.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-arvo 2024-04-24T23:16:10.408 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-isku 2024-04-24T23:16:10.409 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone 2024-04-24T23:16:10.410 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-koneplus 2024-04-24T23:16:10.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-konepure 2024-04-24T23:16:10.412 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-kovaplus 2024-04-24T23:16:10.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-lua 2024-04-24T23:16:10.414 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-pyra 2024-04-24T23:16:11.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-ryos 2024-04-24T23:16:11.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-roccat-savu 2024-04-24T23:16:11.594 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-srws1 2024-04-24T23:16:11.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-hid-wiimote 2024-04-24T23:16:11.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-input-axp-pek 2024-04-24T23:16:11.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-intel-rapid-start 2024-04-24T23:16:11.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-pciback 2024-04-24T23:16:11.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-ppi 2024-04-24T23:16:11.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-samsung-laptop 2024-04-24T23:16:11.601 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-sunxi-sid 2024-04-24T23:16:11.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-tegra-fuse 2024-04-24T23:16:11.603 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-toshiba_acpi 2024-04-24T23:16:11.604 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-wacom 2024-04-24T23:16:11.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkback 2024-04-24T23:16:11.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-driver-xen-blkfront 2024-04-24T23:16:11.607 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-acpi 2024-04-24T23:16:11.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-dmi 2024-04-24T23:16:11.609 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi 2024-04-24T23:16:11.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-efi-runtime-map 2024-04-24T23:16:11.611 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-gsmi 2024-04-24T23:16:11.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-log 2024-04-24T23:16:11.613 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-memmap 2024-04-24T23:16:11.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-ofw 2024-04-24T23:16:11.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sfi 2024-04-24T23:16:11.616 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-firmware-sgi_uv 2024-04-24T23:16:11.617 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-ext4 2024-04-24T23:16:11.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-f2fs 2024-04-24T23:16:11.619 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-nilfs2 2024-04-24T23:16:11.620 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-fs-xfs 2024-04-24T23:16:11.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-gpio 2024-04-24T23:16:11.622 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-i2c-bmp085 2024-04-24T23:16:11.623 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ibft 2024-04-24T23:16:11.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-boot_params 2024-04-24T23:16:11.625 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-fscaps 2024-04-24T23:16:11.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-iommu_groups 2024-04-24T23:16:11.627 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-livepatch 2024-04-24T23:16:11.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm 2024-04-24T23:16:11.629 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-hugepages 2024-04-24T23:16:11.630 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-mm-ksm 2024-04-24T23:16:11.631 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-slab 2024-04-24T23:16:11.632 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-uids 2024-04-24T23:16:11.633 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo 2024-04-24T23:16:11.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-memory-page-offline 2024-04-24T23:16:11.635 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-module 2024-04-24T23:16:11.636 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ocfs2 2024-04-24T23:16:11.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-laptop 2024-04-24T23:16:11.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-asus-wmi 2024-04-24T23:16:11.639 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-at91 2024-04-24T23:16:11.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-brcmstb-gisb-arb 2024-04-24T23:16:11.641 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-chipidea-usb-otg 2024-04-24T23:16:11.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-eeepc-laptop 2024-04-24T23:16:11.643 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ideapad-laptop 2024-04-24T23:16:11.644 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-kim 2024-04-24T23:16:11.645 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-msi-laptop 2024-04-24T23:16:11.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-tahvo-usb 2024-04-24T23:16:11.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-platform-ts5500 2024-04-24T23:16:11.648 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-power 2024-04-24T23:16:11.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-pps 2024-04-24T23:16:11.650 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-profiling 2024-04-24T23:16:11.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-ptp 2024-04-24T23:16:11.652 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-tty 2024-04-24T23:16:11.653 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ABI/testing/sysfs-wusb_cbaf 2024-04-24T23:16:11.654 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/BUG-HUNTING 2024-04-24T23:16:11.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/Changes 2024-04-24T23:16:11.656 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/CodeOfConflict 2024-04-24T23:16:11.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/CodingStyle 2024-04-24T23:16:11.658 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DMA-API-HOWTO.txt 2024-04-24T23:16:11.659 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DMA-API.txt 2024-04-24T23:16:11.660 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DMA-ISA-LPC.txt 2024-04-24T23:16:11.661 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DMA-attributes.txt 2024-04-24T23:16:11.662 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/ 2024-04-24T23:16:11.663 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/.gitignore 2024-04-24T23:16:11.664 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/80211.tmpl 2024-04-24T23:16:11.665 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/Makefile 2024-04-24T23:16:11.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/alsa-driver-api.tmpl 2024-04-24T23:16:11.667 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/crypto-API.tmpl 2024-04-24T23:16:11.668 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/debugobjects.tmpl 2024-04-24T23:16:11.669 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/device-drivers.tmpl 2024-04-24T23:16:11.670 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/deviceiobook.tmpl 2024-04-24T23:16:11.671 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/drm.tmpl 2024-04-24T23:16:11.673 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/filesystems.tmpl 2024-04-24T23:16:11.674 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/gadget.tmpl 2024-04-24T23:16:11.675 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/genericirq.tmpl 2024-04-24T23:16:11.676 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/kernel-api.tmpl 2024-04-24T23:16:11.677 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/kernel-hacking.tmpl 2024-04-24T23:16:11.677 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/kernel-locking.tmpl 2024-04-24T23:16:11.679 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/kgdb.tmpl 2024-04-24T23:16:11.680 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/libata.tmpl 2024-04-24T23:16:11.681 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/librs.tmpl 2024-04-24T23:16:11.682 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/lsm.tmpl 2024-04-24T23:16:11.683 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/ 2024-04-24T23:16:11.684 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/Makefile 2024-04-24T23:16:11.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/bayer.png.b64 2024-04-24T23:16:11.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/constraints.png.b64 2024-04-24T23:16:11.686 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/crop.gif.b64 2024-04-24T23:16:11.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ 2024-04-24T23:16:11.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/.gitignore 2024-04-24T23:16:11.689 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/audio.xml 2024-04-24T23:16:11.690 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/ca.xml 2024-04-24T23:16:11.691 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/demux.xml 2024-04-24T23:16:11.692 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbapi.xml 2024-04-24T23:16:11.693 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbproperty.xml 2024-04-24T23:16:11.694 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/dvbstb.pdf 2024-04-24T23:16:11.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/examples.xml 2024-04-24T23:16:11.696 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/frontend.xml 2024-04-24T23:16:11.697 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/intro.xml 2024-04-24T23:16:11.698 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/kdapi.xml 2024-04-24T23:16:11.699 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/net.xml 2024-04-24T23:16:11.700 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvb/video.xml 2024-04-24T23:16:11.701 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/dvbstb.png.b64 2024-04-24T23:16:11.702 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_bt.gif.b64 2024-04-24T23:16:11.704 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/fieldseq_tb.gif.b64 2024-04-24T23:16:11.705 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt.gif.b64 2024-04-24T23:16:11.706 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/nv12mt_example.gif.b64 2024-04-24T23:16:11.707 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/pipeline.png.b64 2024-04-24T23:16:11.708 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/selection.png.b64 2024-04-24T23:16:11.709 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/ 2024-04-24T23:16:11.710 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/.gitignore 2024-04-24T23:16:11.711 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/biblio.xml 2024-04-24T23:16:11.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/capture.c.xml 2024-04-24T23:16:11.713 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/common.xml 2024-04-24T23:16:11.714 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/compat.xml 2024-04-24T23:16:11.716 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/controls.xml 2024-04-24T23:16:11.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/crop.pdf 2024-04-24T23:16:11.719 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-capture.xml 2024-04-24T23:16:11.720 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-codec.xml 2024-04-24T23:16:11.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-effect.xml 2024-04-24T23:16:11.722 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-event.xml 2024-04-24T23:16:11.723 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-osd.xml 2024-04-24T23:16:11.724 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-output.xml 2024-04-24T23:16:11.725 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-overlay.xml 2024-04-24T23:16:11.726 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-radio.xml 2024-04-24T23:16:11.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-raw-vbi.xml 2024-04-24T23:16:11.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-rds.xml 2024-04-24T23:16:11.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sdr.xml 2024-04-24T23:16:11.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-sliced-vbi.xml 2024-04-24T23:16:11.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-subdev.xml 2024-04-24T23:16:11.732 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/dev-teletext.xml 2024-04-24T23:16:11.733 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/driver.xml 2024-04-24T23:16:11.734 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fdl-appendix.xml 2024-04-24T23:16:11.735 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_bt.pdf 2024-04-24T23:16:11.736 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/fieldseq_tb.pdf 2024-04-24T23:16:11.737 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-close.xml 2024-04-24T23:16:11.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-ioctl.xml 2024-04-24T23:16:11.739 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-mmap.xml 2024-04-24T23:16:11.740 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-munmap.xml 2024-04-24T23:16:11.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-open.xml 2024-04-24T23:16:11.742 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-poll.xml 2024-04-24T23:16:11.743 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-read.xml 2024-04-24T23:16:11.744 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-select.xml 2024-04-24T23:16:11.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/func-write.xml 2024-04-24T23:16:11.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/gen-errors.xml 2024-04-24T23:16:11.747 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/io.xml 2024-04-24T23:16:11.748 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/keytable.c.xml 2024-04-24T23:16:11.749 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/libv4l.xml 2024-04-24T23:16:11.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/lirc_device_interface.xml 2024-04-24T23:16:11.751 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-controller.xml 2024-04-24T23:16:11.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-close.xml 2024-04-24T23:16:11.753 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-ioctl.xml 2024-04-24T23:16:11.754 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-func-open.xml 2024-04-24T23:16:11.755 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-device-info.xml 2024-04-24T23:16:11.756 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-entities.xml 2024-04-24T23:16:11.757 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-enum-links.xml 2024-04-24T23:16:11.758 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/media-ioc-setup-link.xml 2024-04-24T23:16:11.759 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pipeline.pdf 2024-04-24T23:16:11.760 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-grey.xml 2024-04-24T23:16:11.761 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-m420.xml 2024-04-24T23:16:11.762 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12.xml 2024-04-24T23:16:11.763 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12m.xml 2024-04-24T23:16:11.764 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv12mt.xml 2024-04-24T23:16:11.765 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16.xml 2024-04-24T23:16:11.766 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv16m.xml 2024-04-24T23:16:11.767 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-nv24.xml 2024-04-24T23:16:11.768 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-rgb.xml 2024-04-24T23:16:11.769 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-packed-yuv.xml 2024-04-24T23:16:11.771 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr16.xml 2024-04-24T23:16:11.771 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sbggr8.xml 2024-04-24T23:16:11.772 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs08.xml 2024-04-24T23:16:11.773 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cs14le.xml 2024-04-24T23:16:11.774 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu08.xml 2024-04-24T23:16:11.775 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-cu16le.xml 2024-04-24T23:16:11.776 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sdr-ru12le.xml 2024-04-24T23:16:11.777 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgbrg8.xml 2024-04-24T23:16:11.778 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-sgrbg8.xml 2024-04-24T23:16:11.779 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10.xml 2024-04-24T23:16:11.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10alaw8.xml 2024-04-24T23:16:11.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10dpcm8.xml 2024-04-24T23:16:11.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb10p.xml 2024-04-24T23:16:11.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb12.xml 2024-04-24T23:16:11.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-srggb8.xml 2024-04-24T23:16:11.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uv8.xml 2024-04-24T23:16:11.786 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-uyvy.xml 2024-04-24T23:16:11.787 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-vyuy.xml 2024-04-24T23:16:11.788 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10.xml 2024-04-24T23:16:11.789 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y10b.xml 2024-04-24T23:16:11.790 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y12.xml 2024-04-24T23:16:11.791 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y16.xml 2024-04-24T23:16:11.792 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-y41p.xml 2024-04-24T23:16:11.793 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv410.xml 2024-04-24T23:16:11.794 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv411p.xml 2024-04-24T23:16:11.795 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420.xml 2024-04-24T23:16:11.796 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv420m.xml 2024-04-24T23:16:11.797 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuv422p.xml 2024-04-24T23:16:11.798 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yuyv.xml 2024-04-24T23:16:11.799 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvu420m.xml 2024-04-24T23:16:11.800 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt-yvyu.xml 2024-04-24T23:16:11.801 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/pixfmt.xml 2024-04-24T23:16:11.802 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/planar-apis.xml 2024-04-24T23:16:11.804 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/remote_controllers.xml 2024-04-24T23:16:11.805 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selection-api.xml 2024-04-24T23:16:11.806 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/selections-common.xml 2024-04-24T23:16:11.807 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-formats.xml 2024-04-24T23:16:11.808 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.dia 2024-04-24T23:16:11.809 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-crop.svg 2024-04-24T23:16:11.810 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.dia 2024-04-24T23:16:11.811 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-full.svg 2024-04-24T23:16:11.812 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.dia 2024-04-24T23:16:11.813 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/subdev-image-processing-scaling-multi-source.svg 2024-04-24T23:16:11.814 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2.xml 2024-04-24T23:16:11.815 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/v4l2grab.c.xml 2024-04-24T23:16:11.816 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_525.pdf 2024-04-24T23:16:11.817 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_625.pdf 2024-04-24T23:16:11.818 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vbi_hsync.pdf 2024-04-24T23:16:11.819 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-create-bufs.xml 2024-04-24T23:16:11.820 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-cropcap.xml 2024-04-24T23:16:11.821 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-chip-info.xml 2024-04-24T23:16:11.822 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dbg-g-register.xml 2024-04-24T23:16:11.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-decoder-cmd.xml 2024-04-24T23:16:11.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dqevent.xml 2024-04-24T23:16:11.824 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-dv-timings-cap.xml 2024-04-24T23:16:11.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-encoder-cmd.xml 2024-04-24T23:16:11.826 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-dv-timings.xml 2024-04-24T23:16:11.827 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-fmt.xml 2024-04-24T23:16:11.828 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-frameintervals.xml 2024-04-24T23:16:11.829 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-framesizes.xml 2024-04-24T23:16:11.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enum-freq-bands.xml 2024-04-24T23:16:11.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudio.xml 2024-04-24T23:16:11.832 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumaudioout.xml 2024-04-24T23:16:11.833 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enuminput.xml 2024-04-24T23:16:11.834 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumoutput.xml 2024-04-24T23:16:11.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-enumstd.xml 2024-04-24T23:16:11.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-expbuf.xml 2024-04-24T23:16:11.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audio.xml 2024-04-24T23:16:11.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-audioout.xml 2024-04-24T23:16:11.839 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-crop.xml 2024-04-24T23:16:11.840 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ctrl.xml 2024-04-24T23:16:11.841 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-dv-timings.xml 2024-04-24T23:16:11.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-edid.xml 2024-04-24T23:16:11.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-enc-index.xml 2024-04-24T23:16:11.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-ext-ctrls.xml 2024-04-24T23:16:11.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fbuf.xml 2024-04-24T23:16:11.846 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-fmt.xml 2024-04-24T23:16:11.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-frequency.xml 2024-04-24T23:16:11.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-input.xml 2024-04-24T23:16:11.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-jpegcomp.xml 2024-04-24T23:16:11.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-modulator.xml 2024-04-24T23:16:11.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-output.xml 2024-04-24T23:16:11.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-parm.xml 2024-04-24T23:16:11.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-priority.xml 2024-04-24T23:16:11.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-selection.xml 2024-04-24T23:16:11.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-sliced-vbi-cap.xml 2024-04-24T23:16:11.856 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-std.xml 2024-04-24T23:16:11.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-g-tuner.xml 2024-04-24T23:16:11.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-log-status.xml 2024-04-24T23:16:11.859 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-overlay.xml 2024-04-24T23:16:11.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-prepare-buf.xml 2024-04-24T23:16:11.861 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-qbuf.xml 2024-04-24T23:16:11.862 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-query-dv-timings.xml 2024-04-24T23:16:11.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querybuf.xml 2024-04-24T23:16:12.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:12 smithi112 ceph-mon[29186]: pgmap v406: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-24T23:16:12.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:12 smithi150 ceph-mon[32247]: pgmap v406: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-24T23:16:12.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:12 smithi187 ceph-mon[33292]: pgmap v406: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.6 KiB/s wr, 11 op/s 2024-04-24T23:16:13.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querycap.xml 2024-04-24T23:16:13.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-queryctrl.xml 2024-04-24T23:16:13.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-querystd.xml 2024-04-24T23:16:13.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-reqbufs.xml 2024-04-24T23:16:13.005 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-s-hw-freq-seek.xml 2024-04-24T23:16:13.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-streamon.xml 2024-04-24T23:16:13.007 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-interval.xml 2024-04-24T23:16:13.008 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-frame-size.xml 2024-04-24T23:16:13.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-enum-mbus-code.xml 2024-04-24T23:16:13.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-crop.xml 2024-04-24T23:16:13.011 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-fmt.xml 2024-04-24T23:16:13.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-frame-interval.xml 2024-04-24T23:16:13.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subdev-g-selection.xml 2024-04-24T23:16:13.014 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/v4l/vidioc-subscribe-event.xml 2024-04-24T23:16:13.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_525.gif.b64 2024-04-24T23:16:13.016 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_625.gif.b64 2024-04-24T23:16:13.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media/vbi_hsync.gif.b64 2024-04-24T23:16:13.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/media_api.tmpl 2024-04-24T23:16:13.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/mtdnand.tmpl 2024-04-24T23:16:13.020 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/networking.tmpl 2024-04-24T23:16:13.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/rapidio.tmpl 2024-04-24T23:16:13.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/regulator.tmpl 2024-04-24T23:16:13.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/s390-drivers.tmpl 2024-04-24T23:16:13.025 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/scsi.tmpl 2024-04-24T23:16:13.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/sh.tmpl 2024-04-24T23:16:13.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/stylesheet.xsl 2024-04-24T23:16:13.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/tracepoint.tmpl 2024-04-24T23:16:13.029 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/uio-howto.tmpl 2024-04-24T23:16:13.030 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/usb.tmpl 2024-04-24T23:16:13.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/w1.tmpl 2024-04-24T23:16:13.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/writing-an-alsa-driver.tmpl 2024-04-24T23:16:13.035 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/writing_musb_glue_layer.tmpl 2024-04-24T23:16:13.036 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/writing_usb_driver.tmpl 2024-04-24T23:16:13.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/DocBook/z8530book.tmpl 2024-04-24T23:16:13.038 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/ 2024-04-24T23:16:13.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1024x768.S 2024-04-24T23:16:13.040 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1280x1024.S 2024-04-24T23:16:13.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1600x1200.S 2024-04-24T23:16:13.042 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1680x1050.S 2024-04-24T23:16:13.043 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/1920x1080.S 2024-04-24T23:16:13.044 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/800x600.S 2024-04-24T23:16:13.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/HOWTO.txt 2024-04-24T23:16:13.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/Makefile 2024-04-24T23:16:13.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/edid.S 2024-04-24T23:16:13.047 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/EDID/hex 2024-04-24T23:16:13.048 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/HOWTO 2024-04-24T23:16:13.049 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/IPMI.txt 2024-04-24T23:16:13.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/IRQ-affinity.txt 2024-04-24T23:16:13.051 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/IRQ-domain.txt 2024-04-24T23:16:13.052 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/IRQ.txt 2024-04-24T23:16:13.053 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/Intel-IOMMU.txt 2024-04-24T23:16:13.054 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/Makefile 2024-04-24T23:16:13.055 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ManagementStyle 2024-04-24T23:16:13.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/ 2024-04-24T23:16:13.057 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/00-INDEX 2024-04-24T23:16:13.058 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/MSI-HOWTO.txt 2024-04-24T23:16:13.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/PCIEBUS-HOWTO.txt 2024-04-24T23:16:13.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/pci-error-recovery.txt 2024-04-24T23:16:13.061 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/pci-iov-howto.txt 2024-04-24T23:16:13.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/pci.txt 2024-04-24T23:16:13.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/PCI/pcieaer-howto.txt 2024-04-24T23:16:13.064 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/ 2024-04-24T23:16:13.065 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/00-INDEX 2024-04-24T23:16:13.066 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/NMI-RCU.txt 2024-04-24T23:16:13.067 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/RTFP.txt 2024-04-24T23:16:13.068 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/UP.txt 2024-04-24T23:16:13.069 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/arrayRCU.txt 2024-04-24T23:16:13.070 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/checklist.txt 2024-04-24T23:16:13.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/listRCU.txt 2024-04-24T23:16:13.072 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/lockdep-splat.txt 2024-04-24T23:16:13.073 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/lockdep.txt 2024-04-24T23:16:13.074 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rcu.txt 2024-04-24T23:16:13.075 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rcu_dereference.txt 2024-04-24T23:16:13.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rcubarrier.txt 2024-04-24T23:16:13.077 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rculist_nulls.txt 2024-04-24T23:16:13.078 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/rcuref.txt 2024-04-24T23:16:13.079 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/stallwarn.txt 2024-04-24T23:16:13.079 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/torture.txt 2024-04-24T23:16:13.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/trace.txt 2024-04-24T23:16:13.081 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/RCU/whatisRCU.txt 2024-04-24T23:16:13.082 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SAK.txt 2024-04-24T23:16:13.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SM501.txt 2024-04-24T23:16:13.084 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SecurityBugs 2024-04-24T23:16:13.085 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SubmitChecklist 2024-04-24T23:16:13.086 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SubmittingDrivers 2024-04-24T23:16:13.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/SubmittingPatches 2024-04-24T23:16:13.088 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/VGA-softcursor.txt 2024-04-24T23:16:13.089 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/ 2024-04-24T23:16:13.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/.gitignore 2024-04-24T23:16:13.092 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/Makefile 2024-04-24T23:16:13.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/cgroupstats.txt 2024-04-24T23:16:13.094 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/delay-accounting.txt 2024-04-24T23:16:13.095 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/getdelays.c 2024-04-24T23:16:13.096 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/taskstats-struct.txt 2024-04-24T23:16:13.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/accounting/taskstats.txt 2024-04-24T23:16:13.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/ 2024-04-24T23:16:13.098 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/apei/ 2024-04-24T23:16:13.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/apei/einj.txt 2024-04-24T23:16:13.100 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/apei/output_format.txt 2024-04-24T23:16:13.101 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/debug.txt 2024-04-24T23:16:13.102 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/dsdt-override.txt 2024-04-24T23:16:13.103 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/enumeration.txt 2024-04-24T23:16:13.104 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/gpio-properties.txt 2024-04-24T23:16:13.105 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/initrd_table_override.txt 2024-04-24T23:16:13.106 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/method-customizing.txt 2024-04-24T23:16:13.107 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/method-tracing.txt 2024-04-24T23:16:13.108 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/namespace.txt 2024-04-24T23:16:13.109 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/scan_handlers.txt 2024-04-24T23:16:13.110 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/acpi/video_extension.txt 2024-04-24T23:16:13.111 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/ 2024-04-24T23:16:13.112 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/aoe.txt 2024-04-24T23:16:13.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/autoload.sh 2024-04-24T23:16:13.114 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/status.sh 2024-04-24T23:16:13.115 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/todo.txt 2024-04-24T23:16:13.115 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/udev-install.sh 2024-04-24T23:16:13.116 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/aoe/udev.txt 2024-04-24T23:16:13.117 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/applying-patches.txt 2024-04-24T23:16:13.118 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/ 2024-04-24T23:16:13.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/00-INDEX 2024-04-24T23:16:13.120 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Atmel/ 2024-04-24T23:16:13.121 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Atmel/README 2024-04-24T23:16:13.122 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Booting 2024-04-24T23:16:13.122 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/CCN.txt 2024-04-24T23:16:13.123 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/IXP4xx 2024-04-24T23:16:13.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Interrupts 2024-04-24T23:16:13.125 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Makefile 2024-04-24T23:16:13.126 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Marvell/ 2024-04-24T23:16:13.127 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Marvell/README 2024-04-24T23:16:13.128 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Netwinder 2024-04-24T23:16:13.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/OMAP/ 2024-04-24T23:16:13.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/OMAP/DSS 2024-04-24T23:16:13.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/OMAP/omap_pm 2024-04-24T23:16:13.131 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Porting 2024-04-24T23:16:13.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/README 2024-04-24T23:16:13.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/ 2024-04-24T23:16:13.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/ADSBitsy 2024-04-24T23:16:13.134 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Assabet 2024-04-24T23:16:13.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Brutus 2024-04-24T23:16:13.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/CERF 2024-04-24T23:16:13.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/FreeBird 2024-04-24T23:16:13.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsClient 2024-04-24T23:16:13.139 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/GraphicsMaster 2024-04-24T23:16:13.140 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/HUW_WEBPANEL 2024-04-24T23:16:13.141 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Itsy 2024-04-24T23:16:13.142 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/LART 2024-04-24T23:16:13.143 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/PLEB 2024-04-24T23:16:13.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Pangolin 2024-04-24T23:16:13.145 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Tifon 2024-04-24T23:16:13.147 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Victor 2024-04-24T23:16:13.148 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/Yopy 2024-04-24T23:16:13.150 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/empeg 2024-04-24T23:16:13.151 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/nanoEngine 2024-04-24T23:16:13.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SA1100/serial_UART 2024-04-24T23:16:13.154 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/ 2024-04-24T23:16:13.155 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/.gitignore 2024-04-24T23:16:13.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/Makefile 2024-04-24T23:16:13.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/vrl4.c 2024-04-24T23:16:13.160 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-mmcif.txt 2024-04-24T23:16:13.161 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SH-Mobile/zboot-rom-sdhi.txt 2024-04-24T23:16:13.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SPEAr/ 2024-04-24T23:16:13.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/SPEAr/overview.txt 2024-04-24T23:16:13.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/ 2024-04-24T23:16:13.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/CPUfreq.txt 2024-04-24T23:16:13.168 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/EB2410ITX.txt 2024-04-24T23:16:13.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/GPIO.txt 2024-04-24T23:16:13.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/H1940.txt 2024-04-24T23:16:13.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/NAND.txt 2024-04-24T23:16:13.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Overview.txt 2024-04-24T23:16:13.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2412.txt 2024-04-24T23:16:13.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/S3C2413.txt 2024-04-24T23:16:13.178 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/SMDK2440.txt 2024-04-24T23:16:13.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/Suspend.txt 2024-04-24T23:16:13.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung-S3C24XX/USB-Host.txt 2024-04-24T23:16:13.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung/ 2024-04-24T23:16:13.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung/GPIO.txt 2024-04-24T23:16:13.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung/Overview.txt 2024-04-24T23:16:13.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Samsung/clksrc-change-registers.awk 2024-04-24T23:16:13.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/Setup 2024-04-24T23:16:13.189 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/VFP/ 2024-04-24T23:16:13.191 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/VFP/release-notes.txt 2024-04-24T23:16:13.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/cluster-pm-race-avoidance.txt 2024-04-24T23:16:13.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/firmware.txt 2024-04-24T23:16:13.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/kernel_mode_neon.txt 2024-04-24T23:16:13.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/kernel_user_helpers.txt 2024-04-24T23:16:13.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/mem_alignment 2024-04-24T23:16:13.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/memory.txt 2024-04-24T23:16:13.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/msm/ 2024-04-24T23:16:13.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/msm/gpiomux.txt 2024-04-24T23:16:13.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/ 2024-04-24T23:16:13.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/NOTES 2024-04-24T23:16:13.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/README 2024-04-24T23:16:13.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/README.FPE 2024-04-24T23:16:13.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/nwfpe/TODO 2024-04-24T23:16:13.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/pxa/ 2024-04-24T23:16:13.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/pxa/mfp.txt 2024-04-24T23:16:13.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/ 2024-04-24T23:16:13.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/overview.txt 2024-04-24T23:16:13.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/stih407-overview.txt 2024-04-24T23:16:13.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/stih415-overview.txt 2024-04-24T23:16:13.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/stih416-overview.txt 2024-04-24T23:16:13.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sti/stih418-overview.txt 2024-04-24T23:16:13.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sunxi/ 2024-04-24T23:16:13.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sunxi/README 2024-04-24T23:16:13.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/sunxi/clocks.txt 2024-04-24T23:16:13.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/swp_emulation 2024-04-24T23:16:13.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/tcm.txt 2024-04-24T23:16:13.225 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/uefi.txt 2024-04-24T23:16:13.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm/vlocks.txt 2024-04-24T23:16:13.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/ 2024-04-24T23:16:13.228 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/booting.txt 2024-04-24T23:16:13.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/legacy_instructions.txt 2024-04-24T23:16:13.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/memory.txt 2024-04-24T23:16:13.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/arm64/tagged-pointers.txt 2024-04-24T23:16:13.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/assoc_array.txt 2024-04-24T23:16:13.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/atomic_ops.txt 2024-04-24T23:16:13.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/ 2024-04-24T23:16:13.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/.gitignore 2024-04-24T23:16:13.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/Makefile 2024-04-24T23:16:13.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b 2024-04-24T23:16:13.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/cfag12864b-example.c 2024-04-24T23:16:13.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/auxdisplay/ks0108 2024-04-24T23:16:13.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/backlight/ 2024-04-24T23:16:13.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/backlight/lp855x-driver.txt 2024-04-24T23:16:13.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bad_memory.txt 2024-04-24T23:16:13.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/basic_profiling.txt 2024-04-24T23:16:13.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bcache.txt 2024-04-24T23:16:13.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/binfmt_misc.txt 2024-04-24T23:16:13.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/ 2024-04-24T23:16:13.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/00-INDEX 2024-04-24T23:16:13.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/Makefile 2024-04-24T23:16:13.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/bfin-gpio-notes.txt 2024-04-24T23:16:13.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/bfin-spi-notes.txt 2024-04-24T23:16:13.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blackfin/gptimers-example.c 2024-04-24T23:16:13.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/ 2024-04-24T23:16:13.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/00-INDEX 2024-04-24T23:16:13.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/biodoc.txt 2024-04-24T23:16:13.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/biovecs.txt 2024-04-24T23:16:13.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/capability.txt 2024-04-24T23:16:13.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/cfq-iosched.txt 2024-04-24T23:16:13.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/cmdline-partition.txt 2024-04-24T23:16:13.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/data-integrity.txt 2024-04-24T23:16:13.256 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/deadline-iosched.txt 2024-04-24T23:16:13.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/ioprio.txt 2024-04-24T23:16:13.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/null_blk.txt 2024-04-24T23:16:13.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/queue-sysfs.txt 2024-04-24T23:16:13.260 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/request.txt 2024-04-24T23:16:13.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/stat.txt 2024-04-24T23:16:13.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/switching-sched.txt 2024-04-24T23:16:13.263 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/block/writeback_cache_control.txt 2024-04-24T23:16:13.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/ 2024-04-24T23:16:13.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/00-INDEX 2024-04-24T23:16:13.266 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/README.DAC960 2024-04-24T23:16:13.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/cciss.txt 2024-04-24T23:16:13.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/cpqarray.txt 2024-04-24T23:16:13.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/ 2024-04-24T23:16:13.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-8.3-data-packets.svg 2024-04-24T23:16:13.270 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/DRBD-data-packets.svg 2024-04-24T23:16:13.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/README.txt 2024-04-24T23:16:13.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/conn-states-8.dot 2024-04-24T23:16:13.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/data-structure-v9.txt 2024-04-24T23:16:13.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/disk-states-8.dot 2024-04-24T23:16:13.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/drbd-connection-state-overview.dot 2024-04-24T23:16:13.275 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/drbd/node-states-8.dot 2024-04-24T23:16:13.276 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/floppy.txt 2024-04-24T23:16:13.277 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/mflash.txt 2024-04-24T23:16:13.278 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/nbd.txt 2024-04-24T23:16:13.279 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/paride.txt 2024-04-24T23:16:13.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/ramdisk.txt 2024-04-24T23:16:13.281 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/blockdev/zram.txt 2024-04-24T23:16:13.282 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/braille-console.txt 2024-04-24T23:16:13.283 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bt8xxgpio.txt 2024-04-24T23:16:13.284 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/btmrvl.txt 2024-04-24T23:16:13.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bus-devices/ 2024-04-24T23:16:13.286 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bus-devices/ti-gpmc.txt 2024-04-24T23:16:13.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/bus-virt-phys-mapping.txt 2024-04-24T23:16:13.288 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cachetlb.txt 2024-04-24T23:16:13.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/ 2024-04-24T23:16:13.290 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/00-INDEX 2024-04-24T23:16:13.291 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/Makefile 2024-04-24T23:16:13.292 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/cdrom-standard.tex 2024-04-24T23:16:13.293 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/ide-cd 2024-04-24T23:16:13.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cdrom/packet-writing.txt 2024-04-24T23:16:13.295 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/ 2024-04-24T23:16:13.296 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/00-INDEX 2024-04-24T23:16:13.296 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/blkio-controller.txt 2024-04-24T23:16:13.298 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/cgroups.txt 2024-04-24T23:16:13.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/cpuacct.txt 2024-04-24T23:16:13.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/cpusets.txt 2024-04-24T23:16:13.300 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/devices.txt 2024-04-24T23:16:13.301 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/freezer-subsystem.txt 2024-04-24T23:16:13.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/hugetlb.txt 2024-04-24T23:16:13.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/memcg_test.txt 2024-04-24T23:16:13.303 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/memory.txt 2024-04-24T23:16:13.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/net_cls.txt 2024-04-24T23:16:13.305 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/net_prio.txt 2024-04-24T23:16:13.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cgroups/unified-hierarchy.txt 2024-04-24T23:16:13.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/circular-buffers.txt 2024-04-24T23:16:13.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/clk.txt 2024-04-24T23:16:13.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/coccinelle.txt 2024-04-24T23:16:13.309 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/ 2024-04-24T23:16:13.310 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/.gitignore 2024-04-24T23:16:13.311 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/Makefile 2024-04-24T23:16:13.312 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/cn_test.c 2024-04-24T23:16:13.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/connector.txt 2024-04-24T23:16:13.314 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/connector/ucon.c 2024-04-24T23:16:13.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/console/ 2024-04-24T23:16:13.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/console/console.txt 2024-04-24T23:16:13.316 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/ 2024-04-24T23:16:13.317 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/amd-powernow.txt 2024-04-24T23:16:13.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/boost.txt 2024-04-24T23:16:13.319 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/core.txt 2024-04-24T23:16:13.320 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/cpu-drivers.txt 2024-04-24T23:16:13.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-nforce2.txt 2024-04-24T23:16:13.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/cpufreq-stats.txt 2024-04-24T23:16:13.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/governors.txt 2024-04-24T23:16:13.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/index.txt 2024-04-24T23:16:13.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/intel-pstate.txt 2024-04-24T23:16:13.325 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/pcc-cpufreq.txt 2024-04-24T23:16:13.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-freq/user-guide.txt 2024-04-24T23:16:13.327 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-hotplug.txt 2024-04-24T23:16:13.328 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpu-load.txt 2024-04-24T23:16:13.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/ 2024-04-24T23:16:13.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/core.txt 2024-04-24T23:16:13.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/driver.txt 2024-04-24T23:16:13.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/governor.txt 2024-04-24T23:16:13.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cpuidle/sysfs.txt 2024-04-24T23:16:13.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cputopology.txt 2024-04-24T23:16:13.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crc32.txt 2024-04-24T23:16:13.339 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cris/ 2024-04-24T23:16:13.341 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/cris/README 2024-04-24T23:16:13.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/ 2024-04-24T23:16:13.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/api-intro.txt 2024-04-24T23:16:13.345 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/asymmetric-keys.txt 2024-04-24T23:16:13.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/async-tx-api.txt 2024-04-24T23:16:13.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/crypto-API-userspace.txt 2024-04-24T23:16:13.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/crypto/descore-readme.txt 2024-04-24T23:16:13.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dcdbas.txt 2024-04-24T23:16:13.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/debugging-modules.txt 2024-04-24T23:16:13.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/debugging-via-ohci1394.txt 2024-04-24T23:16:13.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dell_rbu.txt 2024-04-24T23:16:13.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/ 2024-04-24T23:16:13.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/1.Intro 2024-04-24T23:16:13.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/2.Process 2024-04-24T23:16:13.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/3.Early-stage 2024-04-24T23:16:13.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/4.Coding 2024-04-24T23:16:13.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/5.Posting 2024-04-24T23:16:13.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/6.Followthrough 2024-04-24T23:16:13.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/7.AdvancedTopics 2024-04-24T23:16:13.369 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/development-process/8.Conclusion 2024-04-24T23:16:13.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/ 2024-04-24T23:16:13.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/cache-policies.txt 2024-04-24T23:16:13.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/cache.txt 2024-04-24T23:16:13.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/delay.txt 2024-04-24T23:16:13.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-crypt.txt 2024-04-24T23:16:13.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-flakey.txt 2024-04-24T23:16:13.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-io.txt 2024-04-24T23:16:13.381 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-log.txt 2024-04-24T23:16:13.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-queue-length.txt 2024-04-24T23:16:13.384 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-raid.txt 2024-04-24T23:16:13.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-service-time.txt 2024-04-24T23:16:13.387 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/dm-uevent.txt 2024-04-24T23:16:13.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/era.txt 2024-04-24T23:16:13.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/kcopyd.txt 2024-04-24T23:16:13.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/linear.txt 2024-04-24T23:16:13.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/persistent-data.txt 2024-04-24T23:16:13.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/snapshot.txt 2024-04-24T23:16:13.396 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/statistics.txt 2024-04-24T23:16:13.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/striped.txt 2024-04-24T23:16:13.399 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/switch.txt 2024-04-24T23:16:13.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/thin-provisioning.txt 2024-04-24T23:16:13.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/verity.txt 2024-04-24T23:16:13.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/device-mapper/zero.txt 2024-04-24T23:16:13.405 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devices.txt 2024-04-24T23:16:13.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/ 2024-04-24T23:16:13.408 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/00-INDEX 2024-04-24T23:16:13.410 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ 2024-04-24T23:16:13.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ABI.txt 2024-04-24T23:16:13.412 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/ 2024-04-24T23:16:13.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/interrupts.txt 2024-04-24T23:16:13.415 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arc/pmu.txt 2024-04-24T23:16:13.416 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ 2024-04-24T23:16:13.418 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/adapteva.txt 2024-04-24T23:16:13.419 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/ 2024-04-24T23:16:13.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.txt 2024-04-24T23:16:13.422 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-sdram-edac.txt 2024-04-24T23:16:13.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/altera/socfpga-system.txt 2024-04-24T23:16:13.425 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/amlogic.txt 2024-04-24T23:16:13.427 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arch_timer.txt 2024-04-24T23:16:13.428 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/arm-boards 2024-04-24T23:16:13.430 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp-pmsu.txt 2024-04-24T23:16:13.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-370-xp.txt 2024-04-24T23:16:13.433 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-375.txt 2024-04-24T23:16:13.434 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-380-mpcore-soc-ctrl.txt 2024-04-24T23:16:13.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-38x.txt 2024-04-24T23:16:13.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armada-cpu-reset.txt 2024-04-24T23:16:13.439 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/armadeus.txt 2024-04-24T23:16:13.440 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-at91.txt 2024-04-24T23:16:13.442 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/atmel-pmc.txt 2024-04-24T23:16:13.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/axxia.txt 2024-04-24T23:16:13.445 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/ 2024-04-24T23:16:13.446 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm11351.txt 2024-04-24T23:16:13.448 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm21664.txt 2024-04-24T23:16:13.449 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/bcm63138.txt 2024-04-24T23:16:13.451 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/brcm,bcm11351-cpu-method 2024-04-24T23:16:13.452 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/cygnus.txt 2024-04-24T23:16:13.454 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-resetmgr.txt 2024-04-24T23:16:13.456 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-timer.txt 2024-04-24T23:16:13.457 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm/kona-wdt.txt 2024-04-24T23:16:13.458 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm2835.txt 2024-04-24T23:16:13.460 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/bcm4708.txt 2024-04-24T23:16:13.461 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/brcm-brcmstb.txt 2024-04-24T23:16:13.463 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda.txt 2024-04-24T23:16:13.464 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/ 2024-04-24T23:16:13.465 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/combophy.txt 2024-04-24T23:16:13.466 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/l2ecc.txt 2024-04-24T23:16:13.467 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/calxeda/mem-ctrlr.txt 2024-04-24T23:16:13.468 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cavium-thunder.txt 2024-04-24T23:16:13.469 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cci.txt 2024-04-24T23:16:13.470 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ccn.txt 2024-04-24T23:16:13.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coherency-fabric.txt 2024-04-24T23:16:13.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/coresight.txt 2024-04-24T23:16:13.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/ 2024-04-24T23:16:13.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpu-enable-method/marvell,berlin-smp 2024-04-24T23:16:13.475 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/cpus.txt 2024-04-24T23:16:13.476 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci.txt 2024-04-24T23:16:13.477 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/ 2024-04-24T23:16:13.478 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/davinci/cp-intc.txt 2024-04-24T23:16:13.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/digicolor.txt 2024-04-24T23:16:13.480 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/ 2024-04-24T23:16:13.480 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/power_domain.txt 2024-04-24T23:16:13.481 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/exynos/smp-sysram.txt 2024-04-24T23:16:13.482 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/ 2024-04-24T23:16:13.483 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/firmware/tlm,trusted-foundations.txt 2024-04-24T23:16:13.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fsl.txt 2024-04-24T23:16:13.485 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/fw-cfg.txt 2024-04-24T23:16:13.485 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/geniatech.txt 2024-04-24T23:16:13.486 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic-v3.txt 2024-04-24T23:16:13.487 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/gic.txt 2024-04-24T23:16:13.488 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/global_timer.txt 2024-04-24T23:16:13.489 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/ 2024-04-24T23:16:13.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/hisilicon/hisilicon.txt 2024-04-24T23:16:13.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/idle-states.txt 2024-04-24T23:16:13.491 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/insignal-boards.txt 2024-04-24T23:16:13.492 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/ 2024-04-24T23:16:13.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/keystone/keystone.txt 2024-04-24T23:16:13.494 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/kirkwood.txt 2024-04-24T23:16:13.495 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/l2cc.txt 2024-04-24T23:16:13.496 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx-mic.txt 2024-04-24T23:16:13.497 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/lpc32xx.txt 2024-04-24T23:16:13.498 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,berlin.txt 2024-04-24T23:16:13.498 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,dove.txt 2024-04-24T23:16:13.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/marvell,kirkwood.txt 2024-04-24T23:16:13.500 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek.txt 2024-04-24T23:16:13.501 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/ 2024-04-24T23:16:13.502 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mediatek/mediatek,sysirq.txt 2024-04-24T23:16:13.503 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/moxart.txt 2024-04-24T23:16:13.504 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/ 2024-04-24T23:16:13.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/feroceon.txt 2024-04-24T23:16:13.506 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/intc.txt 2024-04-24T23:16:13.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/mrvl.txt 2024-04-24T23:16:13.508 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/tauros2.txt 2024-04-24T23:16:13.509 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mrvl/timer.txt 2024-04-24T23:16:13.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ 2024-04-24T23:16:13.511 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,kpss-acc.txt 2024-04-24T23:16:13.512 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/qcom,saw2.txt 2024-04-24T23:16:13.512 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/ssbi.txt 2024-04-24T23:16:13.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/msm/timer.txt 2024-04-24T23:16:13.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/mvebu-system-controller.txt 2024-04-24T23:16:13.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/nspire.txt 2024-04-24T23:16:13.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/olimex.txt 2024-04-24T23:16:13.517 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/ 2024-04-24T23:16:13.518 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/counter.txt 2024-04-24T23:16:13.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/crossbar.txt 2024-04-24T23:16:13.520 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dmm.txt 2024-04-24T23:16:13.521 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/dsp.txt 2024-04-24T23:16:13.522 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/intc.txt 2024-04-24T23:16:13.523 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/iva.txt 2024-04-24T23:16:13.524 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/l3-noc.txt 2024-04-24T23:16:13.524 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/mpu.txt 2024-04-24T23:16:13.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/omap.txt 2024-04-24T23:16:13.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/prcm.txt 2024-04-24T23:16:13.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/omap/timer.txt 2024-04-24T23:16:13.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/picoxcell.txt 2024-04-24T23:16:13.529 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/pmu.txt 2024-04-24T23:16:13.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/primecell.txt 2024-04-24T23:16:13.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/psci.txt 2024-04-24T23:16:13.532 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip.txt 2024-04-24T23:16:13.533 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/ 2024-04-24T23:16:13.534 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu-sram.txt 2024-04-24T23:16:13.535 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/pmu.txt 2024-04-24T23:16:13.536 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rockchip/smp-sram.txt 2024-04-24T23:16:13.537 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/rtsm-dcscb.txt 2024-04-24T23:16:13.538 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung-boards.txt 2024-04-24T23:16:13.539 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/ 2024-04-24T23:16:13.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-adc.txt 2024-04-24T23:16:13.541 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/exynos-chipid.txt 2024-04-24T23:16:13.542 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/interrupt-combiner.txt 2024-04-24T23:16:13.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/pmu.txt 2024-04-24T23:16:13.544 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/samsung/sysreg.txt 2024-04-24T23:16:13.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/shmobile.txt 2024-04-24T23:16:13.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sirf.txt 2024-04-24T23:16:13.547 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-misc.txt 2024-04-24T23:16:13.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear-timer.txt 2024-04-24T23:16:13.549 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear.txt 2024-04-24T23:16:13.550 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/ 2024-04-24T23:16:13.551 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/spear/shirq.txt 2024-04-24T23:16:13.552 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sprd.txt 2024-04-24T23:16:13.553 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-nomadik.txt 2024-04-24T23:16:13.554 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ste-u300.txt 2024-04-24T23:16:13.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sti.txt 2024-04-24T23:16:13.556 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/sunxi.txt 2024-04-24T23:16:13.557 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra.txt 2024-04-24T23:16:13.558 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/ 2024-04-24T23:16:13.559 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-ahb.txt 2024-04-24T23:16:13.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-emc.txt 2024-04-24T23:16:13.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-flowctrl.txt 2024-04-24T23:16:13.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-mc.txt 2024-04-24T23:16:13.562 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra20-pmc.txt 2024-04-24T23:16:13.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/tegra/nvidia,tegra30-mc.txt 2024-04-24T23:16:13.564 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/topology.txt 2024-04-24T23:16:13.565 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/twd.txt 2024-04-24T23:16:13.566 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/ 2024-04-24T23:16:13.567 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/ux500/power_domain.txt 2024-04-24T23:16:13.568 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-fpga-irq.txt 2024-04-24T23:16:13.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/versatile-sysreg.txt 2024-04-24T23:16:13.570 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-scc.txt 2024-04-24T23:16:13.571 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress-sysreg.txt 2024-04-24T23:16:13.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vexpress.txt 2024-04-24T23:16:13.573 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vic.txt 2024-04-24T23:16:13.574 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500.txt 2024-04-24T23:16:13.575 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/ 2024-04-24T23:16:13.576 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-intc.txt 2024-04-24T23:16:13.817 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-pmc.txt 2024-04-24T23:16:13.818 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/vt8500/via,vt8500-timer.txt 2024-04-24T23:16:13.819 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xen.txt 2024-04-24T23:16:13.820 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/arm/xilinx.txt 2024-04-24T23:16:13.821 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ 2024-04-24T23:16:13.822 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-platform.txt 2024-04-24T23:16:13.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/ahci-st.txt 2024-04-24T23:16:13.824 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/apm-xgene.txt 2024-04-24T23:16:13.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/atmel-at91_cf.txt 2024-04-24T23:16:13.826 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/cavium-compact-flash.txt 2024-04-24T23:16:13.827 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/exynos-sata.txt 2024-04-24T23:16:13.827 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/fsl-sata.txt 2024-04-24T23:16:13.829 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-pata.txt 2024-04-24T23:16:13.829 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/imx-sata.txt 2024-04-24T23:16:13.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/marvell.txt 2024-04-24T23:16:13.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/pata-arasan.txt 2024-04-24T23:16:13.832 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/qcom-sata.txt 2024-04-24T23:16:13.833 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_highbank.txt 2024-04-24T23:16:13.834 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/sata_rcar.txt 2024-04-24T23:16:13.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ata/tegra-sata.txt 2024-04-24T23:16:13.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/btmrvl.txt 2024-04-24T23:16:13.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ 2024-04-24T23:16:13.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/bcma.txt 2024-04-24T23:16:13.839 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/brcm,gisb-arb.txt 2024-04-24T23:16:13.840 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/imx-weim.txt 2024-04-24T23:16:13.841 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/mvebu-mbus.txt 2024-04-24T23:16:13.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/omap-ocp2scp.txt 2024-04-24T23:16:13.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/bus/ti-gpmc.txt 2024-04-24T23:16:13.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/ 2024-04-24T23:16:13.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/clocks.txt 2024-04-24T23:16:13.846 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/dscr.txt 2024-04-24T23:16:13.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/emifa.txt 2024-04-24T23:16:13.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/interrupt.txt 2024-04-24T23:16:13.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/soc.txt 2024-04-24T23:16:13.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/c6x/timer64.txt 2024-04-24T23:16:13.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/chosen.txt 2024-04-24T23:16:13.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ 2024-04-24T23:16:13.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/alphascale,acc.txt 2024-04-24T23:16:13.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/altr_socfpga.txt 2024-04-24T23:16:13.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/arm-integrator.txt 2024-04-24T23:16:13.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/at91-clock.txt 2024-04-24T23:16:13.856 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/axi-clkgen.txt 2024-04-24T23:16:13.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-cygnus-clock.txt 2024-04-24T23:16:13.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/bcm-kona-clock.txt 2024-04-24T23:16:13.859 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/calxeda.txt 2024-04-24T23:16:13.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-exynos-audss.txt 2024-04-24T23:16:13.861 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-palmas-clk32kg-clocks.txt 2024-04-24T23:16:13.862 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clk-s5pv210-audss.txt 2024-04-24T23:16:13.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clock-bindings.txt 2024-04-24T23:16:13.864 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/clps711x-clock.txt 2024-04-24T23:16:13.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/efm32-clock.txt 2024-04-24T23:16:13.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/emev2-clock.txt 2024-04-24T23:16:13.866 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos3250-clock.txt 2024-04-24T23:16:13.867 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4-clock.txt 2024-04-24T23:16:13.868 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos4415-clock.txt 2024-04-24T23:16:13.869 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5250-clock.txt 2024-04-24T23:16:13.870 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5260-clock.txt 2024-04-24T23:16:13.871 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5410-clock.txt 2024-04-24T23:16:13.872 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5420-clock.txt 2024-04-24T23:16:13.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos5440-clock.txt 2024-04-24T23:16:13.874 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/exynos7-clock.txt 2024-04-24T23:16:13.875 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-clock.txt 2024-04-24T23:16:13.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/fixed-factor-clock.txt 2024-04-24T23:16:13.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/gpio-gate-clock.txt 2024-04-24T23:16:13.878 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hi3620-clock.txt 2024-04-24T23:16:13.879 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/hix5hd2-clock.txt 2024-04-24T23:16:13.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx1-clock.txt 2024-04-24T23:16:13.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx21-clock.txt 2024-04-24T23:16:13.881 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx23-clock.txt 2024-04-24T23:16:13.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx25-clock.txt 2024-04-24T23:16:13.883 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx27-clock.txt 2024-04-24T23:16:13.884 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx28-clock.txt 2024-04-24T23:16:13.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx31-clock.txt 2024-04-24T23:16:13.886 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx35-clock.txt 2024-04-24T23:16:13.887 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx5-clock.txt 2024-04-24T23:16:13.888 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6q-clock.txt 2024-04-24T23:16:13.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sl-clock.txt 2024-04-24T23:16:13.890 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/imx6sx-clock.txt 2024-04-24T23:16:13.890 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-gate.txt 2024-04-24T23:16:13.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/keystone-pll.txt 2024-04-24T23:16:13.892 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/lsi,axm5516-clks.txt 2024-04-24T23:16:13.893 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,mmp2.txt 2024-04-24T23:16:13.894 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa168.txt 2024-04-24T23:16:13.895 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/marvell,pxa910.txt 2024-04-24T23:16:13.896 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77686.txt 2024-04-24T23:16:13.897 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/maxim,max77802.txt 2024-04-24T23:16:13.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/moxa,moxart-clock.txt 2024-04-24T23:16:13.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-core-clock.txt 2024-04-24T23:16:13.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-corediv-clock.txt 2024-04-24T23:16:13.900 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-cpu-clock.txt 2024-04-24T23:16:13.901 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/mvebu-gated-clock.txt 2024-04-24T23:16:13.902 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nspire-clock.txt 2024-04-24T23:16:13.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra114-car.txt 2024-04-24T23:16:13.904 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra124-car.txt 2024-04-24T23:16:13.905 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra20-car.txt 2024-04-24T23:16:13.906 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/nvidia,tegra30-car.txt 2024-04-24T23:16:13.907 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/prima2-clock.txt 2024-04-24T23:16:13.908 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/pxa-clock.txt 2024-04-24T23:16:13.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,gcc.txt 2024-04-24T23:16:13.910 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,lcc.txt 2024-04-24T23:16:13.911 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qcom,mmcc.txt 2024-04-24T23:16:13.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/qoriq-clock.txt 2024-04-24T23:16:13.913 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-div6-clocks.txt 2024-04-24T23:16:13.913 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,cpg-mstp-clocks.txt 2024-04-24T23:16:13.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a73a4-cpg-clocks.txt 2024-04-24T23:16:13.915 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7740-cpg-clocks.txt 2024-04-24T23:16:13.916 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,r8a7779-cpg-clocks.txt 2024-04-24T23:16:13.917 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rcar-gen2-cpg-clocks.txt 2024-04-24T23:16:13.918 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,rz-cpg-clocks.txt 2024-04-24T23:16:13.919 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/renesas,sh73a0-cpg-clocks.txt 2024-04-24T23:16:13.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3188-cru.txt 2024-04-24T23:16:13.921 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip,rk3288-cru.txt 2024-04-24T23:16:13.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/rockchip.txt 2024-04-24T23:16:13.923 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2410-clock.txt 2024-04-24T23:16:13.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2412-clock.txt 2024-04-24T23:16:13.925 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c2443-clock.txt 2024-04-24T23:16:13.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s3c64xx-clock.txt 2024-04-24T23:16:13.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/samsung,s5pv210-clock.txt 2024-04-24T23:16:13.928 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si5351.txt 2024-04-24T23:16:13.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/silabs,si570.txt 2024-04-24T23:16:13.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st,nomadik.txt 2024-04-24T23:16:13.931 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/ 2024-04-24T23:16:13.932 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-divmux.txt 2024-04-24T23:16:13.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-mux.txt 2024-04-24T23:16:13.934 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-pll.txt 2024-04-24T23:16:13.935 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-prediv.txt 2024-04-24T23:16:13.936 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen-vcc.txt 2024-04-24T23:16:13.937 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,clkgen.txt 2024-04-24T23:16:13.938 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,flexgen.txt 2024-04-24T23:16:13.939 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/st/st,quadfs.txt 2024-04-24T23:16:13.940 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ste-u300-syscon-clock.txt 2024-04-24T23:16:13.941 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/sunxi.txt 2024-04-24T23:16:13.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti,cdce706.txt 2024-04-24T23:16:13.943 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti-keystone-pllctrl.txt 2024-04-24T23:16:13.944 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/ 2024-04-24T23:16:13.945 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/apll.txt 2024-04-24T23:16:13.945 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/autoidle.txt 2024-04-24T23:16:13.946 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/clockdomain.txt 2024-04-24T23:16:13.947 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/composite.txt 2024-04-24T23:16:13.948 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/divider.txt 2024-04-24T23:16:13.949 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dpll.txt 2024-04-24T23:16:13.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/dra7-atl.txt 2024-04-24T23:16:13.951 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fapll.txt 2024-04-24T23:16:13.952 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/fixed-factor-clock.txt 2024-04-24T23:16:13.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/gate.txt 2024-04-24T23:16:13.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/interface.txt 2024-04-24T23:16:13.954 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/ti/mux.txt 2024-04-24T23:16:13.955 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vf610-clock.txt 2024-04-24T23:16:13.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/vt8500.txt 2024-04-24T23:16:13.957 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/xgene.txt 2024-04-24T23:16:13.958 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/clock/zynq-7000.txt 2024-04-24T23:16:13.959 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/ 2024-04-24T23:16:13.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/arm_big_little_dt.txt 2024-04-24T23:16:13.961 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-dt.txt 2024-04-24T23:16:13.962 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-exynos5440.txt 2024-04-24T23:16:13.963 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/cpufreq/cpufreq-spear.txt 2024-04-24T23:16:13.964 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/ 2024-04-24T23:16:13.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/amd-ccp.txt 2024-04-24T23:16:13.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/atmel-crypto.txt 2024-04-24T23:16:13.966 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-dcp.txt 2024-04-24T23:16:13.967 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-imx-sahara.txt 2024-04-24T23:16:13.968 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec2.txt 2024-04-24T23:16:13.969 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec4.txt 2024-04-24T23:16:13.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/fsl-sec6.txt 2024-04-24T23:16:13.971 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/mv_cesa.txt 2024-04-24T23:16:13.972 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-aes.txt 2024-04-24T23:16:13.973 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-des.txt 2024-04-24T23:16:13.974 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/omap-sham.txt 2024-04-24T23:16:13.975 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/picochip-spacc.txt 2024-04-24T23:16:13.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/qcom-qce.txt 2024-04-24T23:16:13.977 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/crypto/samsung-sss.txt 2024-04-24T23:16:13.978 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/ 2024-04-24T23:16:13.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/ 2024-04-24T23:16:13.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/devfreq/event/exynos-ppmu.txt 2024-04-24T23:16:13.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ 2024-04-24T23:16:13.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/arm-pl330.txt 2024-04-24T23:16:13.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-dma.txt 2024-04-24T23:16:13.983 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/atmel-xdma.txt 2024-04-24T23:16:13.984 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/bcm2835-dma.txt 2024-04-24T23:16:13.985 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/dma.txt 2024-04-24T23:16:13.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-edma.txt 2024-04-24T23:16:13.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-dma.txt 2024-04-24T23:16:13.987 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-imx-sdma.txt 2024-04-24T23:16:13.988 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/fsl-mxs-dma.txt 2024-04-24T23:16:13.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/img-mdc-dma.txt 2024-04-24T23:16:13.990 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/k3dma.txt 2024-04-24T23:16:13.991 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mmp-dma.txt 2024-04-24T23:16:13.992 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/moxa,moxart-dma.txt 2024-04-24T23:16:13.993 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mpc512x-dma.txt 2024-04-24T23:16:13.994 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/mv-xor.txt 2024-04-24T23:16:13.995 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/nbpfaxi.txt 2024-04-24T23:16:13.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_adm.txt 2024-04-24T23:16:13.997 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/qcom_bam_dma.txt 2024-04-24T23:16:13.998 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/rcar-audmapp.txt 2024-04-24T23:16:13.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/renesas,rcar-dmac.txt 2024-04-24T23:16:14.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/shdma.txt 2024-04-24T23:16:14.001 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sirfsoc-dma.txt 2024-04-24T23:16:14.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/snps-dma.txt 2024-04-24T23:16:14.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-coh901318.txt 2024-04-24T23:16:14.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ste-dma40.txt 2024-04-24T23:16:14.005 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/sun6i-dma.txt 2024-04-24T23:16:14.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/tegra20-apbdma.txt 2024-04-24T23:16:14.007 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/ti-edma.txt 2024-04-24T23:16:14.008 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/ 2024-04-24T23:16:14.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_dma.txt 2024-04-24T23:16:14.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/dma/xilinx/xilinx_vdma.txt 2024-04-24T23:16:14.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/ 2024-04-24T23:16:14.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/ 2024-04-24T23:16:14.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/armada/marvell,dove-lcd.txt 2024-04-24T23:16:14.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/ 2024-04-24T23:16:14.014 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/atmel/hlcdc-dc.txt 2024-04-24T23:16:14.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/ 2024-04-24T23:16:14.016 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/bridge/dw_hdmi.txt 2024-04-24T23:16:14.016 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/ 2024-04-24T23:16:14.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/i2c/tda998x.txt 2024-04-24T23:16:14.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ 2024-04-24T23:16:14.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/fsl-imx-drm.txt 2024-04-24T23:16:14.020 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/hdmi.txt 2024-04-24T23:16:14.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/imx/ldb.txt 2024-04-24T23:16:14.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/ 2024-04-24T23:16:14.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/gpu.txt 2024-04-24T23:16:14.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/hdmi.txt 2024-04-24T23:16:14.025 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/msm/mdp.txt 2024-04-24T23:16:14.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/ 2024-04-24T23:16:14.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/panel.txt 2024-04-24T23:16:14.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/slave.txt 2024-04-24T23:16:14.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tfp410.txt 2024-04-24T23:16:14.029 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/drm/tilcdc/tilcdc.txt 2024-04-24T23:16:14.030 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/eeprom.txt 2024-04-24T23:16:14.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/ 2024-04-24T23:16:14.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-palmas.txt 2024-04-24T23:16:14.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-rt8973a.txt 2024-04-24T23:16:14.035 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/extcon/extcon-sm5502.txt 2024-04-24T23:16:14.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/ 2024-04-24T23:16:14.038 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/mxsfb.txt 2024-04-24T23:16:14.040 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fb/sm501fb.txt 2024-04-24T23:16:14.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/ 2024-04-24T23:16:14.043 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fpga/altera-socfpga-fpga-mgr.txt 2024-04-24T23:16:14.044 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/ 2024-04-24T23:16:14.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/fuse/nvidia,tegra20-fuse.txt 2024-04-24T23:16:14.047 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/ 2024-04-24T23:16:14.049 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/8xxx_gpio.txt 2024-04-24T23:16:14.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/abilis,tb10x-gpio.txt 2024-04-24T23:16:14.052 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cavium-octeon-gpio.txt 2024-04-24T23:16:14.053 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/cirrus,clps711x-mctrl-gpio.txt 2024-04-24T23:16:14.055 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fsl-imx-gpio.txt 2024-04-24T23:16:14.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/fujitsu,mb86s70-gpio.txt 2024-04-24T23:16:14.057 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74x164.txt 2024-04-24T23:16:14.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-74xx-mmio.txt 2024-04-24T23:16:14.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-adnp.txt 2024-04-24T23:16:14.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-bcm-kona.txt 2024-04-24T23:16:14.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-clps711x.txt 2024-04-24T23:16:14.065 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-davinci.txt 2024-04-24T23:16:14.066 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-dsp-keystone.txt 2024-04-24T23:16:14.068 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-fan.txt 2024-04-24T23:16:14.069 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-grgpio.txt 2024-04-24T23:16:14.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-lp3943.txt 2024-04-24T23:16:14.072 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-max732x.txt 2024-04-24T23:16:14.074 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mcp23s08.txt 2024-04-24T23:16:14.075 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mm-lantiq.txt 2024-04-24T23:16:14.077 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-msm.txt 2024-04-24T23:16:14.078 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mvebu.txt 2024-04-24T23:16:14.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-mxs.txt 2024-04-24T23:16:14.081 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-nmk.txt 2024-04-24T23:16:14.082 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-omap.txt 2024-04-24T23:16:14.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-palmas.txt 2024-04-24T23:16:14.084 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pca953x.txt 2024-04-24T23:16:14.085 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-pcf857x.txt 2024-04-24T23:16:14.086 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-poweroff.txt 2024-04-24T23:16:14.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-restart.txt 2024-04-24T23:16:14.088 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-samsung.txt 2024-04-24T23:16:14.089 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stericsson-coh901.txt 2024-04-24T23:16:14.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stmpe.txt 2024-04-24T23:16:14.091 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-stp-xway.txt 2024-04-24T23:16:14.092 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-sx150x.txt 2024-04-24T23:16:14.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-twl4030.txt 2024-04-24T23:16:14.094 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090-pdc.txt 2024-04-24T23:16:14.095 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-tz1090.txt 2024-04-24T23:16:14.095 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-vf610.txt 2024-04-24T23:16:14.096 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene-sb.txt 2024-04-24T23:16:14.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xgene.txt 2024-04-24T23:16:14.098 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-xilinx.txt 2024-04-24T23:16:14.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zevio.txt 2024-04-24T23:16:14.100 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio-zynq.txt 2024-04-24T23:16:14.101 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio.txt 2024-04-24T23:16:14.102 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_atmel.txt 2024-04-24T23:16:14.103 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/gpio_lpc32xx.txt 2024-04-24T23:16:14.104 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/moxa,moxart-gpio.txt 2024-04-24T23:16:14.105 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/mrvl-gpio.txt 2024-04-24T23:16:14.106 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/nvidia,tegra20-gpio.txt 2024-04-24T23:16:14.107 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/pl061-gpio.txt 2024-04-24T23:16:14.108 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/renesas,gpio-rcar.txt 2024-04-24T23:16:14.109 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt 2024-04-24T23:16:14.110 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/sodaville.txt 2024-04-24T23:16:14.111 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpio/spear_spics.txt 2024-04-24T23:16:14.112 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/ 2024-04-24T23:16:14.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,gk20a.txt 2024-04-24T23:16:14.114 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/nvidia,tegra20-host1x.txt 2024-04-24T23:16:14.115 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-g2d.txt 2024-04-24T23:16:14.116 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/samsung-rotator.txt 2024-04-24T23:16:14.117 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/gpu/st,stih4xx.txt 2024-04-24T23:16:14.118 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/graph.txt 2024-04-24T23:16:14.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/ 2024-04-24T23:16:14.120 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hid/hid-over-i2c.txt 2024-04-24T23:16:14.121 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/ 2024-04-24T23:16:14.122 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/client-devices.txt 2024-04-24T23:16:14.123 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/nokia-modem.txt 2024-04-24T23:16:14.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hsi/omap-ssi.txt 2024-04-24T23:16:14.125 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ 2024-04-24T23:16:14.126 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ads1015.txt 2024-04-24T23:16:14.126 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/g762.txt 2024-04-24T23:16:14.128 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ibmpowernv.txt 2024-04-24T23:16:14.128 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/lm90.txt 2024-04-24T23:16:14.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ltc2978.txt 2024-04-24T23:16:14.131 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/ntc_thermistor.txt 2024-04-24T23:16:14.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/pwm-fan.txt 2024-04-24T23:16:14.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwmon/vexpress.txt 2024-04-24T23:16:14.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/ 2024-04-24T23:16:14.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/atmel-trng.txt 2024-04-24T23:16:14.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/omap_rng.txt 2024-04-24T23:16:14.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/hwrng/timeriomem_rng.txt 2024-04-24T23:16:14.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ 2024-04-24T23:16:14.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,bcm2835-i2c.txt 2024-04-24T23:16:14.139 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/brcm,iproc-i2c.txt 2024-04-24T23:16:14.140 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-arb-gpio-challenge.txt 2024-04-24T23:16:14.141 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-at91.txt 2024-04-24T23:16:14.142 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-axxia.txt 2024-04-24T23:16:14.143 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-bcm-kona.txt 2024-04-24T23:16:14.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cadence.txt 2024-04-24T23:16:14.145 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cbus-gpio.txt 2024-04-24T23:16:14.146 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-cros-ec-tunnel.txt 2024-04-24T23:16:14.147 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-davinci.txt 2024-04-24T23:16:14.148 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-designware.txt 2024-04-24T23:16:14.149 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-efm32.txt 2024-04-24T23:16:14.150 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-exynos5.txt 2024-04-24T23:16:14.151 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-gpio.txt 2024-04-24T23:16:14.152 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-hix5hd2.txt 2024-04-24T23:16:14.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-img-scb.txt 2024-04-24T23:16:14.154 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-imx.txt 2024-04-24T23:16:14.155 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-meson.txt 2024-04-24T23:16:14.156 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mpc.txt 2024-04-24T23:16:14.156 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-gpio.txt 2024-04-24T23:16:14.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pca954x.txt 2024-04-24T23:16:14.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux-pinctrl.txt 2024-04-24T23:16:14.159 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mux.txt 2024-04-24T23:16:14.160 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mv64xxx.txt 2024-04-24T23:16:14.161 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-mxs.txt 2024-04-24T23:16:14.162 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-nomadik.txt 2024-04-24T23:16:14.163 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-ocores.txt 2024-04-24T23:16:14.164 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-octeon.txt 2024-04-24T23:16:14.165 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-omap.txt 2024-04-24T23:16:14.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-opal.txt 2024-04-24T23:16:14.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pnx.txt 2024-04-24T23:16:14.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa-pci-ce4100.txt 2024-04-24T23:16:14.168 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-pxa.txt 2024-04-24T23:16:14.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rcar.txt 2024-04-24T23:16:14.170 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-riic.txt 2024-04-24T23:16:14.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-rk3x.txt 2024-04-24T23:16:14.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-s3c2410.txt 2024-04-24T23:16:14.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sh_mobile.txt 2024-04-24T23:16:14.174 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sirf.txt 2024-04-24T23:16:14.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st-ddci2c.txt 2024-04-24T23:16:14.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-st.txt 2024-04-24T23:16:14.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-sunxi-p2wi.txt 2024-04-24T23:16:14.178 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-versatile.txt 2024-04-24T23:16:14.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-vt8500.txt 2024-04-24T23:16:14.180 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/i2c-xiic.txt 2024-04-24T23:16:14.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina209.txt 2024-04-24T23:16:14.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ina2xx.txt 2024-04-24T23:16:14.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/max6697.txt 2024-04-24T23:16:14.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/nvidia,tegra20-i2c.txt 2024-04-24T23:16:14.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/qcom,i2c-qup.txt 2024-04-24T23:16:14.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/ti,bq32k.txt 2024-04-24T23:16:14.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/i2c/trivial-devices.txt 2024-04-24T23:16:14.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/ 2024-04-24T23:16:14.189 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/ 2024-04-24T23:16:14.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/accel/bma180.txt 2024-04-24T23:16:14.191 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/ 2024-04-24T23:16:14.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/at91_adc.txt 2024-04-24T23:16:14.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/cc10001_adc.txt 2024-04-24T23:16:14.194 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/max1027-adc.txt 2024-04-24T23:16:14.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/nuvoton-nau7802.txt 2024-04-24T23:16:14.196 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-iadc.txt 2024-04-24T23:16:14.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/qcom,spmi-vadc.txt 2024-04-24T23:16:14.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/rockchip-saradc.txt 2024-04-24T23:16:14.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/twl4030-madc.txt 2024-04-24T23:16:14.200 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/vf610-adc.txt 2024-04-24T23:16:14.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/adc/xilinx-xadc.txt 2024-04-24T23:16:14.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ 2024-04-24T23:16:14.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/ad7303.txt 2024-04-24T23:16:14.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/dac/max5821.txt 2024-04-24T23:16:14.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/ 2024-04-24T23:16:14.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/frequency/adf4350.txt 2024-04-24T23:16:14.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/ 2024-04-24T23:16:14.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/humidity/dht11.txt 2024-04-24T23:16:14.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/iio-bindings.txt 2024-04-24T23:16:14.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/ 2024-04-24T23:16:14.210 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/apds9300.txt 2024-04-24T23:16:14.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/cm36651.txt 2024-04-24T23:16:14.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/gp2ap020a00f.txt 2024-04-24T23:16:14.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/light/tsl2563.txt 2024-04-24T23:16:14.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ 2024-04-24T23:16:14.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/ak8975.txt 2024-04-24T23:16:14.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/magnetometer/hmc5843.txt 2024-04-24T23:16:14.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/ 2024-04-24T23:16:14.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/proximity/as3935.txt 2024-04-24T23:16:14.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/sensorhub.txt 2024-04-24T23:16:14.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iio/st-sensors.txt 2024-04-24T23:16:14.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ 2024-04-24T23:16:14.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ads7846.txt 2024-04-24T23:16:14.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/atmel,maxtouch.txt 2024-04-24T23:16:14.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cap11xx.txt 2024-04-24T23:16:14.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/clps711x-keypad.txt 2024-04-24T23:16:14.225 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/cros-ec-keyb.txt 2024-04-24T23:16:14.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/e3x0-button.txt 2024-04-24T23:16:14.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elan_i2c.txt 2024-04-24T23:16:14.228 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/elants_i2c.txt 2024-04-24T23:16:14.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/fsl-mma8450.txt 2024-04-24T23:16:14.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-beeper.txt 2024-04-24T23:16:14.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys-polled.txt 2024-04-24T23:16:14.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-keys.txt 2024-04-24T23:16:14.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/gpio-matrix-keypad.txt 2024-04-24T23:16:14.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/imx-keypad.txt 2024-04-24T23:16:14.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/input-reset.txt 2024-04-24T23:16:14.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/lpc32xx-key.txt 2024-04-24T23:16:14.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/matrix-keymap.txt 2024-04-24T23:16:14.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/nvidia,tegra20-kbc.txt 2024-04-24T23:16:14.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/omap-keypad.txt 2024-04-24T23:16:14.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ps2keyb-mouse-apbps2.txt 2024-04-24T23:16:14.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pwm-beeper.txt 2024-04-24T23:16:14.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/pxa27x-keypad.txt 2024-04-24T23:16:14.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-keypad.txt 2024-04-24T23:16:14.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-pwrkey.txt 2024-04-24T23:16:14.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/qcom,pm8xxx-vib.txt 2024-04-24T23:16:14.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/regulator-haptic.txt 2024-04-24T23:16:14.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/rotary-encoder.txt 2024-04-24T23:16:14.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/samsung-keypad.txt 2024-04-24T23:16:14.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/spear-keyboard.txt 2024-04-24T23:16:14.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/st-keyscan.txt 2024-04-24T23:16:14.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/stmpe-keypad.txt 2024-04-24T23:16:14.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/sun4i-lradc-keys.txt 2024-04-24T23:16:14.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tca8418_keypad.txt 2024-04-24T23:16:14.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv260x.txt 2024-04-24T23:16:14.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,drv2667.txt 2024-04-24T23:16:14.256 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,nspire-keypad.txt 2024-04-24T23:16:14.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/ti,palmas-pwrbutton.txt 2024-04-24T23:16:14.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ 2024-04-24T23:16:14.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/auo_pixcir_ts.txt 2024-04-24T23:16:14.260 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/bu21013.txt 2024-04-24T23:16:14.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/edt-ft5x06.txt 2024-04-24T23:16:14.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/egalax-ts.txt 2024-04-24T23:16:14.263 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/lpc32xx-tsc.txt 2024-04-24T23:16:14.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/mms114.txt 2024-04-24T23:16:14.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/pixcir_i2c_ts.txt 2024-04-24T23:16:14.266 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sitronix-st1232.txt 2024-04-24T23:16:14.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/stmpe.txt 2024-04-24T23:16:14.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/sun4i.txt 2024-04-24T23:16:14.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/ti-tsc-adc.txt 2024-04-24T23:16:14.270 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/touchscreen.txt 2024-04-24T23:16:14.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2005.txt 2024-04-24T23:16:14.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/tsc2007.txt 2024-04-24T23:16:14.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/touchscreen/zforce_ts.txt 2024-04-24T23:16:14.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/tps65218-pwrbutton.txt 2024-04-24T23:16:14.276 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-keypad.txt 2024-04-24T23:16:14.277 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/input/twl4030-pwrbutton.txt 2024-04-24T23:16:14.278 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ 2024-04-24T23:16:14.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:14 smithi112 ceph-mon[29186]: pgmap v407: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-24T23:16:14.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/abilis,tb10x-ictl.txt 2024-04-24T23:16:14.281 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun4i-ic.txt 2024-04-24T23:16:14.282 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/allwinner,sun67i-sc-nmi.txt 2024-04-24T23:16:14.283 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/atmel,aic.txt 2024-04-24T23:16:14.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm2835-armctrl-ic.txt 2024-04-24T23:16:14.286 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,bcm7120-l2-intc.txt 2024-04-24T23:16:14.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/brcm,l2-intc.txt 2024-04-24T23:16:14.288 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/cirrus,clps711x-intc.txt 2024-04-24T23:16:14.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/digicolor-ic.txt 2024-04-24T23:16:14.291 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/interrupts.txt 2024-04-24T23:16:14.292 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/lsi,zevio-intc.txt 2024-04-24T23:16:14.293 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,armada-370-xp-mpic.txt 2024-04-24T23:16:14.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/marvell,orion-intc.txt 2024-04-24T23:16:14.295 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/mips-gic.txt 2024-04-24T23:16:14.297 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/opencores,or1k-pic.txt 2024-04-24T23:16:14.298 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,intc-irqpin.txt 2024-04-24T23:16:14.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/renesas,irqc.txt 2024-04-24T23:16:14.300 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/samsung,s3c24xx-irq.txt 2024-04-24T23:16:14.301 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/snps,dw-apb-ictl.txt 2024-04-24T23:16:14.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,keystone-irq.txt 2024-04-24T23:16:14.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/interrupt-controller/ti,omap-intc-irq.txt 2024-04-24T23:16:14.305 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ 2024-04-24T23:16:14.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/arm,smmu.txt 2024-04-24T23:16:14.307 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/iommu.txt 2024-04-24T23:16:14.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra20-gart.txt 2024-04-24T23:16:14.309 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/nvidia,tegra30-smmu.txt 2024-04-24T23:16:14.311 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/renesas,ipmmu-vmsa.txt 2024-04-24T23:16:14.312 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/rockchip,iommu.txt 2024-04-24T23:16:14.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/samsung,sysmmu.txt 2024-04-24T23:16:14.314 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/iommu/ti,omap-iommu.txt 2024-04-24T23:16:14.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/ 2024-04-24T23:16:14.316 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/common.txt 2024-04-24T23:16:14.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-gpio.txt 2024-04-24T23:16:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:14 smithi150 ceph-mon[32247]: pgmap v407: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-24T23:16:14.319 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp55xx.txt 2024-04-24T23:16:14.320 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-lp8860.txt 2024-04-24T23:16:14.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-ns2.txt 2024-04-24T23:16:14.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/leds-pwm.txt 2024-04-24T23:16:14.323 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/pca963x.txt 2024-04-24T23:16:14.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/register-bit-led.txt 2024-04-24T23:16:14.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/leds/tca6507.txt 2024-04-24T23:16:14.327 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/ 2024-04-24T23:16:14.328 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2-timings.txt 2024-04-24T23:16:14.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/lpddr2/lpddr2.txt 2024-04-24T23:16:14.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/ 2024-04-24T23:16:14.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/altera-mailbox.txt 2024-04-24T23:16:14.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/mailbox.txt 2024-04-24T23:16:14.334 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mailbox/omap-mailbox.txt 2024-04-24T23:16:14.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/marvell.txt 2024-04-24T23:16:14.336 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ 2024-04-24T23:16:14.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/atmel-isi.txt 2024-04-24T23:16:14.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/coda.txt 2024-04-24T23:16:14.339 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-fimc-lite.txt 2024-04-24T23:16:14.340 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos-jpeg-codec.txt 2024-04-24T23:16:14.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos4-fimc-is.txt 2024-04-24T23:16:14.343 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/exynos5-gsc.txt 2024-04-24T23:16:14.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/gpio-ir-receiver.txt 2024-04-24T23:16:14.345 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/hix5hd2-ir.txt 2024-04-24T23:16:14.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ 2024-04-24T23:16:14.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7343.txt 2024-04-24T23:16:14.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/adv7604.txt 2024-04-24T23:16:14.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9m111.txt 2024-04-24T23:16:14.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/mt9p031.txt 2024-04-24T23:16:14.352 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/nokia,smia.txt 2024-04-24T23:16:14.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/ths8200.txt 2024-04-24T23:16:14.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp514x.txt 2024-04-24T23:16:14.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/i2c/tvp7002.txt 2024-04-24T23:16:14.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/img-ir-rev1.txt 2024-04-24T23:16:14.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/meson-ir.txt 2024-04-24T23:16:14.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/pxa-camera.txt 2024-04-24T23:16:14.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/rcar_vin.txt 2024-04-24T23:16:14.361 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/renesas,vsp1.txt 2024-04-24T23:16:14.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/s5p-mfc.txt 2024-04-24T23:16:14.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-fimc.txt 2024-04-24T23:16:14.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-mipi-csis.txt 2024-04-24T23:16:14.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5c73m3.txt 2024-04-24T23:16:14.366 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k5baf.txt 2024-04-24T23:16:14.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/samsung-s5k6a3.txt 2024-04-24T23:16:14.369 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sh_mobile_ceu.txt 2024-04-24T23:16:14.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/si4713.txt 2024-04-24T23:16:14.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/st-rc.txt 2024-04-24T23:16:14.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/sunxi-ir.txt 2024-04-24T23:16:14.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/ti-am437x-vpfe.txt 2024-04-24T23:16:14.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/media/video-interfaces.txt 2024-04-24T23:16:14.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ 2024-04-24T23:16:14.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ 2024-04-24T23:16:14.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/fsl/ifc.txt 2024-04-24T23:16:14.379 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-devbus.txt 2024-04-24T23:16:14.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/mvebu-sdram-controller.txt 2024-04-24T23:16:14.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra-mc.txt 2024-04-24T23:16:14.383 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/renesas-memory-controllers.txt 2024-04-24T23:16:14.384 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/synopsys.txt 2024-04-24T23:16:14.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti-aemif.txt 2024-04-24T23:16:14.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/ 2024-04-24T23:16:14.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/memory-controllers/ti/emif.txt 2024-04-24T23:16:14.389 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/ 2024-04-24T23:16:14.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta-intc.txt 2024-04-24T23:16:14.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/meta.txt 2024-04-24T23:16:14.392 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/metag/pdc-intc.txt 2024-04-24T23:16:14.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ 2024-04-24T23:16:14.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/88pm860x.txt 2024-04-24T23:16:14.396 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ab8500.txt 2024-04-24T23:16:14.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/arizona.txt 2024-04-24T23:16:14.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3711.txt 2024-04-24T23:16:14.399 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/as3722.txt 2024-04-24T23:16:14.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-gpbr.txt 2024-04-24T23:16:14.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-hlcdc.txt 2024-04-24T23:16:14.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-matrix.txt 2024-04-24T23:16:14.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/atmel-smc.txt 2024-04-24T23:16:14.405 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bcm590xx.txt 2024-04-24T23:16:14.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/bfticu.txt 2024-04-24T23:16:14.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/cros-ec.txt 2024-04-24T23:16:14.408 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9052-i2c.txt 2024-04-24T23:16:14.410 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9055.txt 2024-04-24T23:16:14.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/da9063.txt 2024-04-24T23:16:14.412 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/hi6421.txt 2024-04-24T23:16:14.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/lp3943.txt 2024-04-24T23:16:14.414 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max14577.txt 2024-04-24T23:16:14.416 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77686.txt 2024-04-24T23:16:14.417 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max77693.txt 2024-04-24T23:16:14.418 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8925.txt 2024-04-24T23:16:14.419 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/max8998.txt 2024-04-24T23:16:14.420 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/mc13xxx.txt 2024-04-24T23:16:14.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-host.txt 2024-04-24T23:16:14.422 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/omap-usb-tll.txt 2024-04-24T23:16:14.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/palmas.txt 2024-04-24T23:16:14.425 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom,spmi-pmic.txt 2024-04-24T23:16:14.426 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-pm8xxx.txt 2024-04-24T23:16:14.427 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qcom-rpm.txt 2024-04-24T23:16:14.428 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/qriox.txt 2024-04-24T23:16:14.430 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rk808.txt 2024-04-24T23:16:14.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/rn5t618.txt 2024-04-24T23:16:14.432 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mpa01.txt 2024-04-24T23:16:14.433 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/s2mps11.txt 2024-04-24T23:16:14.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/stmpe.txt 2024-04-24T23:16:14.436 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/sun6i-prcm.txt 2024-04-24T23:16:14.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/syscon.txt 2024-04-24T23:16:14.438 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tc3589x.txt 2024-04-24T23:16:14.440 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/ti-keystone-devctrl.txt 2024-04-24T23:16:14.441 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps6507x.txt 2024-04-24T23:16:14.442 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/tps65910.txt 2024-04-24T23:16:14.444 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl-familly.txt 2024-04-24T23:16:14.445 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-audio.txt 2024-04-24T23:16:14.446 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl4030-power.txt 2024-04-24T23:16:14.447 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mfd/twl6040.txt 2024-04-24T23:16:14.448 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/ 2024-04-24T23:16:14.450 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/ 2024-04-24T23:16:14.452 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt 2024-04-24T23:16:14.453 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mipi/nvidia,tegra114-mipi.txt 2024-04-24T23:16:14.454 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ 2024-04-24T23:16:14.455 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/ 2024-04-24T23:16:14.457 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bcm3384-intc.txt 2024-04-24T23:16:14.458 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/bmips.txt 2024-04-24T23:16:14.459 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/cm-dsl.txt 2024-04-24T23:16:14.460 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/brcm/usb.txt 2024-04-24T23:16:14.462 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ 2024-04-24T23:16:14.463 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/bootbus.txt 2024-04-24T23:16:14.464 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/cib.txt 2024-04-24T23:16:14.465 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu.txt 2024-04-24T23:16:14.467 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/ciu2.txt 2024-04-24T23:16:14.468 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/dma-engine.txt 2024-04-24T23:16:14.469 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cavium/uctl.txt 2024-04-24T23:16:14.470 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/cpu_irq.txt 2024-04-24T23:16:14.472 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mips/ralink.txt 2024-04-24T23:16:14.473 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ 2024-04-24T23:16:14.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/allwinner,sunxi-sid.txt 2024-04-24T23:16:14.475 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/arm-charlcd.txt 2024-04-24T23:16:14.476 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/at25.txt 2024-04-24T23:16:14.478 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/atmel-ssc.txt 2024-04-24T23:16:14.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/bmp085.txt 2024-04-24T23:16:14.480 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/fsl,qoriq-mc.txt 2024-04-24T23:16:14.482 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ifm-csi.txt 2024-04-24T23:16:14.483 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/lis302.txt 2024-04-24T23:16:14.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/nvidia,tegra20-apbmisc.txt 2024-04-24T23:16:14.486 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/smc.txt 2024-04-24T23:16:14.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:14 smithi187 ceph-mon[33292]: pgmap v407: 129 pgs: 129 active+clean; 687 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-24T23:16:14.487 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/sram.txt 2024-04-24T23:16:14.488 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/misc/ti,dac7512.txt 2024-04-24T23:16:14.489 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ 2024-04-24T23:16:14.491 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/arasan,sdhci.txt 2024-04-24T23:16:14.492 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/atmel-hsmci.txt 2024-04-24T23:16:14.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/brcm,bcm2835-sdhci.txt 2024-04-24T23:16:14.495 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/davinci_mmc.txt 2024-04-24T23:16:14.496 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/exynos-dw-mshc.txt 2024-04-24T23:16:14.497 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-esdhc.txt 2024-04-24T23:16:14.498 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-esdhc.txt 2024-04-24T23:16:14.500 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/fsl-imx-mmc.txt 2024-04-24T23:16:14.501 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/img-dw-mshc.txt 2024-04-24T23:16:14.502 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/k3-dw-mshc.txt 2024-04-24T23:16:14.503 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/kona-sdhci.txt 2024-04-24T23:16:14.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-emmc.txt 2024-04-24T23:16:14.506 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-pwrseq-simple.txt 2024-04-24T23:16:14.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc-spi-slot.txt 2024-04-24T23:16:14.509 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmc.txt 2024-04-24T23:16:14.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mmci.txt 2024-04-24T23:16:14.511 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/moxa,moxart-mmc.txt 2024-04-24T23:16:14.512 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/mxs-mmc.txt 2024-04-24T23:16:14.513 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/nvidia,tegra20-sdhci.txt 2024-04-24T23:16:14.515 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/orion-sdio.txt 2024-04-24T23:16:14.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/pxa-mmc.txt 2024-04-24T23:16:14.517 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/renesas,mmcif.txt 2024-04-24T23:16:14.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/rockchip-dw-mshc.txt 2024-04-24T23:16:14.520 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/samsung-sdhci.txt 2024-04-24T23:16:14.521 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-dove.txt 2024-04-24T23:16:14.522 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-fujitsu.txt 2024-04-24T23:16:14.524 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-msm.txt 2024-04-24T23:16:14.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-pxa.txt 2024-04-24T23:16:14.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-sirf.txt 2024-04-24T23:16:14.527 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-spear.txt 2024-04-24T23:16:14.529 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sdhci-st.txt 2024-04-24T23:16:14.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/socfpga-dw-mshc.txt 2024-04-24T23:16:14.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/sunxi-mmc.txt 2024-04-24T23:16:14.533 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/synopsys-dw-mshc.txt 2024-04-24T23:16:14.534 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap-hsmmc.txt 2024-04-24T23:16:14.535 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/ti-omap.txt 2024-04-24T23:16:14.536 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/tmio_mmc.txt 2024-04-24T23:16:14.538 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/usdhi6rol0.txt 2024-04-24T23:16:14.539 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mmc/vt8500-sdmmc.txt 2024-04-24T23:16:14.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/ 2024-04-24T23:16:14.541 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/arm-versatile.txt 2024-04-24T23:16:14.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-dataflash.txt 2024-04-24T23:16:14.544 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/atmel-nand.txt 2024-04-24T23:16:14.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/davinci-nand.txt 2024-04-24T23:16:14.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/denali-nand.txt 2024-04-24T23:16:14.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/diskonchip.txt 2024-04-24T23:16:14.549 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/elm.txt 2024-04-24T23:16:14.550 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/flctl-nand.txt 2024-04-24T23:16:14.551 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-quadspi.txt 2024-04-24T23:16:14.553 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsl-upm-nand.txt 2024-04-24T23:16:14.554 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/fsmc-nand.txt 2024-04-24T23:16:14.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpio-control-nand.txt 2024-04-24T23:16:14.556 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nand.txt 2024-04-24T23:16:14.558 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-nor.txt 2024-04-24T23:16:14.559 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmc-onenand.txt 2024-04-24T23:16:14.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/gpmi-nand.txt 2024-04-24T23:16:14.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/hisi504-nand.txt 2024-04-24T23:16:14.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-mlc.txt 2024-04-24T23:16:14.564 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/lpc32xx-slc.txt 2024-04-24T23:16:14.565 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/m25p80.txt 2024-04-24T23:16:14.567 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mtd-physmap.txt 2024-04-24T23:16:14.568 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/mxc-nand.txt 2024-04-24T23:16:14.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/nand.txt 2024-04-24T23:16:14.570 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/orion-nand.txt 2024-04-24T23:16:14.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/partition.txt 2024-04-24T23:16:14.573 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/pxa3xx-nand.txt 2024-04-24T23:16:14.574 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/spear_smi.txt 2024-04-24T23:16:14.576 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/st-fsm.txt 2024-04-24T23:16:14.577 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/mtd/sunxi-nand.txt 2024-04-24T23:16:14.578 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ 2024-04-24T23:16:14.580 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-emac.txt 2024-04-24T23:16:14.581 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun4i-mdio.txt 2024-04-24T23:16:14.582 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/allwinner,sun7i-a20-gmac.txt 2024-04-24T23:16:14.583 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/altera_tse.txt 2024-04-24T23:16:14.585 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe-phy.txt 2024-04-24T23:16:14.586 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/amd-xgbe.txt 2024-04-24T23:16:14.587 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/apm-xgene-enet.txt 2024-04-24T23:16:14.588 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/arc_emac.txt 2024-04-24T23:16:14.590 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcm87xx.txt 2024-04-24T23:16:14.591 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-bcmgenet.txt 2024-04-24T23:16:14.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-mdio-unimac.txt 2024-04-24T23:16:14.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-sf2.txt 2024-04-24T23:16:14.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/broadcom-systemport.txt 2024-04-24T23:16:14.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/calxeda-xgmac.txt 2024-04-24T23:16:14.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/ 2024-04-24T23:16:14.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/atmel-can.txt 2024-04-24T23:16:14.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/c_can.txt 2024-04-24T23:16:14.601 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/cc770.txt 2024-04-24T23:16:14.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/fsl-flexcan.txt 2024-04-24T23:16:14.603 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/grcan.txt 2024-04-24T23:16:14.604 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/m_can.txt 2024-04-24T23:16:14.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/microchip,mcp251x.txt 2024-04-24T23:16:14.607 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/mpc5xxx-mscan.txt 2024-04-24T23:16:14.609 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/rcar_can.txt 2024-04-24T23:16:14.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/sja1000.txt 2024-04-24T23:16:14.611 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/can/xilinx_can.txt 2024-04-24T23:16:14.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mdio.txt 2024-04-24T23:16:14.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-mix.txt 2024-04-24T23:16:14.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cavium-pip.txt 2024-04-24T23:16:14.616 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cdns-emac.txt 2024-04-24T23:16:14.617 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw-phy-sel.txt 2024-04-24T23:16:14.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/cpsw.txt 2024-04-24T23:16:14.620 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davicom-dm9000.txt 2024-04-24T23:16:14.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci-mdio.txt 2024-04-24T23:16:14.622 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/davinci_emac.txt 2024-04-24T23:16:14.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/ 2024-04-24T23:16:14.625 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/dsa/dsa.txt 2024-04-24T23:16:14.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/emac_rockchip.txt 2024-04-24T23:16:14.627 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ethernet.txt 2024-04-24T23:16:14.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fixed-link.txt 2024-04-24T23:16:14.630 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-fec.txt 2024-04-24T23:16:14.631 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/fsl-tsec-phy.txt 2024-04-24T23:16:14.632 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/gpmc-eth.txt 2024-04-24T23:16:14.633 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hip04-net.txt 2024-04-24T23:16:14.635 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/hisilicon-hix5hd2-gmac.txt 2024-04-24T23:16:14.636 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/ 2024-04-24T23:16:14.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/at86rf230.txt 2024-04-24T23:16:14.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/ieee802154/cc2520.txt 2024-04-24T23:16:14.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/keystone-netcp.txt 2024-04-24T23:16:14.641 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/lpc-eth.txt 2024-04-24T23:16:14.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/macb.txt 2024-04-24T23:16:14.643 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt 2024-04-24T23:16:14.645 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-mdio.txt 2024-04-24T23:16:14.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-orion-net.txt 2024-04-24T23:16:14.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pp2.txt 2024-04-24T23:16:14.648 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/marvell-pxa168.txt 2024-04-24T23:16:14.650 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-gpio.txt 2024-04-24T23:16:14.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-gpio.txt 2024-04-24T23:16:14.652 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux-mmioreg.txt 2024-04-24T23:16:14.653 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/mdio-mux.txt 2024-04-24T23:16:14.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/meson-dwmac.txt 2024-04-24T23:16:14.656 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ks8851.txt 2024-04-24T23:16:14.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel-ksz90x1.txt 2024-04-24T23:16:14.658 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/micrel.txt 2024-04-24T23:16:14.659 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/moxa,moxart-mac.txt 2024-04-24T23:16:14.661 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/ 2024-04-24T23:16:14.662 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/pn544.txt 2024-04-24T23:16:14.663 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfca.txt 2024-04-24T23:16:14.664 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/st21nfcb.txt 2024-04-24T23:16:14.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/nfc/trf7970a.txt 2024-04-24T23:16:14.667 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/opencores-ethoc.txt 2024-04-24T23:16:14.668 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/phy.txt 2024-04-24T23:16:14.669 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/qca-qca7000-spi.txt 2024-04-24T23:16:14.670 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/rockchip-dwmac.txt 2024-04-24T23:16:14.672 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/samsung-sxgbe.txt 2024-04-24T23:16:14.673 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sh_eth.txt 2024-04-24T23:16:14.674 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc-lan91c111.txt 2024-04-24T23:16:14.675 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/smsc911x.txt 2024-04-24T23:16:14.677 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/socfpga-dwmac.txt 2024-04-24T23:16:14.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/sti-dwmac.txt 2024-04-24T23:16:14.679 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/stmmac.txt 2024-04-24T23:16:14.680 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-rhine.txt 2024-04-24T23:16:14.682 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/via-velocity.txt 2024-04-24T23:16:14.683 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ 2024-04-24T23:16:14.684 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/brcm,bcm43xx-fmac.txt 2024-04-24T23:16:14.686 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt 2024-04-24T23:16:14.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/net/wireless/ti,wl1251.txt 2024-04-24T23:16:14.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/ 2024-04-24T23:16:14.689 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/nios2.txt 2024-04-24T23:16:14.690 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nios2/timer.txt 2024-04-24T23:16:14.691 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/ 2024-04-24T23:16:14.693 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/nvec/nvidia,nvec.txt 2024-04-24T23:16:14.694 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/open-pic.txt 2024-04-24T23:16:14.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/ 2024-04-24T23:16:14.696 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101aw03.txt 2024-04-24T23:16:14.698 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b101xtn01.txt 2024-04-24T23:16:14.699 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b116xw03.txt 2024-04-24T23:16:14.700 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133htn01.txt 2024-04-24T23:16:14.701 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/auo,b133xtn01.txt 2024-04-24T23:16:14.703 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/avic,tm070ddh03.txt 2024-04-24T23:16:14.704 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wa01a.txt 2024-04-24T23:16:14.705 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/chunghwa,claa101wb03.txt 2024-04-24T23:16:14.706 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et057090dhu.txt 2024-04-24T23:16:14.708 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,et070080dh6.txt 2024-04-24T23:16:14.709 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/edt,etm0700g0dh6.txt 2024-04-24T23:16:14.710 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/foxlink,fl500wvr00-a0t.txt 2024-04-24T23:16:14.711 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/giantplus,gpg482739qs5.txt 2024-04-24T23:16:14.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hannstar,hsd070pww1.txt 2024-04-24T23:16:14.714 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/hit,tx23d38vm0caa.txt 2024-04-24T23:16:14.715 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,g121i1-l01.txt 2024-04-24T23:16:14.716 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n116bge.txt 2024-04-24T23:16:14.717 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/innolux,n156bge-l21.txt 2024-04-24T23:16:14.719 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,ld070wx3-sl01.txt 2024-04-24T23:16:14.720 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lh500wx1-sd03.txt 2024-04-24T23:16:14.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/lg,lp129qe.txt 2024-04-24T23:16:14.722 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/panasonic,vvx10f004b00.txt 2024-04-24T23:16:14.724 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ld9040.txt 2024-04-24T23:16:14.725 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,ltn101nt05.txt 2024-04-24T23:16:14.726 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/samsung,s6e8aa0.txt 2024-04-24T23:16:14.727 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/sharp,lq101r1sx01.txt 2024-04-24T23:16:14.729 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/panel/simple-panel.txt 2024-04-24T23:16:14.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ 2024-04-24T23:16:14.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/83xx-512x-pci.txt 2024-04-24T23:16:14.732 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/designware-pcie.txt 2024-04-24T23:16:14.733 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,imx6q-pcie.txt 2024-04-24T23:16:14.735 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/fsl,pci.txt 2024-04-24T23:16:14.736 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/host-generic-pci.txt 2024-04-24T23:16:14.737 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/layerscape-pci.txt 2024-04-24T23:16:14.739 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/mvebu-pci.txt 2024-04-24T23:16:14.740 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/nvidia,tegra20-pcie.txt 2024-04-24T23:16:14.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-keystone.txt 2024-04-24T23:16:14.743 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci-rcar-gen2.txt 2024-04-24T23:16:14.744 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/pci.txt 2024-04-24T23:16:14.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ralink,rt3883-pci.txt 2024-04-24T23:16:14.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/rcar-pci.txt 2024-04-24T23:16:14.747 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/samsung,exynos5440-pcie.txt 2024-04-24T23:16:14.749 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/spear13xx-pcie.txt 2024-04-24T23:16:14.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/ti-pci.txt 2024-04-24T23:16:14.751 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/v3-v360epc-pci.txt 2024-04-24T23:16:14.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/versatile.txt 2024-04-24T23:16:14.754 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xgene-pci.txt 2024-04-24T23:16:14.755 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pci/xilinx-pcie.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/apm-xgene-phy.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/bcm-phy.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-sata-phy.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/berlin-usb-phy.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/hix5hd2-phy.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-bindings.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy28lp.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-miphy365x.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-mvebu.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih407-usb.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/phy-stih41x-usb.txt 2024-04-24T23:16:14.783 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-apq8064-sata-phy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-dwc3-usb-phy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/qcom-ipq806x-sata-phy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rcar-gen2-phy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/rockchip-usb-phy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/samsung-phy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/st-spear-miphy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/sun4i-usb-phy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/phy/ti-phy.txt 2024-04-24T23:16:14.784 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ 2024-04-24T23:16:14.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/abilis,tb10x-iomux.txt 2024-04-24T23:16:14.785 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/allwinner,sunxi-pinctrl.txt 2024-04-24T23:16:14.786 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/atmel,at91-pinctrl.txt 2024-04-24T23:16:14.787 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm11351-pinctrl.txt 2024-04-24T23:16:14.788 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/brcm,bcm2835-gpio.txt 2024-04-24T23:16:14.789 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx-pinctrl.txt 2024-04-24T23:16:14.791 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx25-pinctrl.txt 2024-04-24T23:16:14.792 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx27-pinctrl.txt 2024-04-24T23:16:14.793 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx35-pinctrl.txt 2024-04-24T23:16:14.794 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx51-pinctrl.txt 2024-04-24T23:16:14.796 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx53-pinctrl.txt 2024-04-24T23:16:14.797 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6dl-pinctrl.txt 2024-04-24T23:16:14.798 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6q-pinctrl.txt 2024-04-24T23:16:14.801 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sl-pinctrl.txt 2024-04-24T23:16:14.802 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,imx6sx-pinctrl.txt 2024-04-24T23:16:14.802 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,mxs-pinctrl.txt 2024-04-24T23:16:14.803 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/fsl,vf610-pinctrl.txt 2024-04-24T23:16:14.804 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pdc-pinctrl.txt 2024-04-24T23:16:14.806 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/img,tz1090-pinctrl.txt 2024-04-24T23:16:14.807 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,falcon-pinumx.txt 2024-04-24T23:16:14.808 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/lantiq,xway-pinumx.txt 2024-04-24T23:16:14.809 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-370-pinctrl.txt 2024-04-24T23:16:14.811 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-375-pinctrl.txt 2024-04-24T23:16:14.812 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-38x-pinctrl.txt 2024-04-24T23:16:14.813 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,armada-xp-pinctrl.txt 2024-04-24T23:16:14.815 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,dove-pinctrl.txt 2024-04-24T23:16:14.816 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,kirkwood-pinctrl.txt 2024-04-24T23:16:14.817 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,mvebu-pinctrl.txt 2024-04-24T23:16:14.818 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/marvell,orion-pinctrl.txt 2024-04-24T23:16:14.819 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/meson,pinctrl.txt 2024-04-24T23:16:14.820 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra114-pinmux.txt 2024-04-24T23:16:14.821 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-pinmux.txt 2024-04-24T23:16:14.822 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra124-xusb-padctl.txt 2024-04-24T23:16:14.823 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra20-pinmux.txt 2024-04-24T23:16:14.824 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/nvidia,tegra30-pinmux.txt 2024-04-24T23:16:14.825 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt 2024-04-24T23:16:14.826 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 2024-04-24T23:16:14.827 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-single.txt 2024-04-24T23:16:14.828 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-sirf.txt 2024-04-24T23:16:14.829 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-st.txt 2024-04-24T23:16:14.830 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl-vt8500.txt 2024-04-24T23:16:14.831 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/pinctrl_spear.txt 2024-04-24T23:16:14.832 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8064-pinctrl.txt 2024-04-24T23:16:14.833 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,apq8084-pinctrl.txt 2024-04-24T23:16:14.834 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,ipq8064-pinctrl.txt 2024-04-24T23:16:14.835 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8916-pinctrl.txt 2024-04-24T23:16:14.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8960-pinctrl.txt 2024-04-24T23:16:14.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,msm8974-pinctrl.txt 2024-04-24T23:16:14.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-gpio.txt 2024-04-24T23:16:14.839 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/qcom,pmic-mpp.txt 2024-04-24T23:16:14.840 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/renesas,pfc-pinctrl.txt 2024-04-24T23:16:14.841 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/rockchip,pinctrl.txt 2024-04-24T23:16:14.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt 2024-04-24T23:16:14.843 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,abx500.txt 2024-04-24T23:16:14.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ste,nomadik.txt 2024-04-24T23:16:14.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/ti,omap-pinctrl.txt 2024-04-24T23:16:14.846 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pinctrl/xlnx,zynq-pinctrl.txt 2024-04-24T23:16:14.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ 2024-04-24T23:16:14.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/bq2415x.txt 2024-04-24T23:16:14.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/isp1704.txt 2024-04-24T23:16:14.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/ltc2941.txt 2024-04-24T23:16:14.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/opp.txt 2024-04-24T23:16:14.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power-controller.txt 2024-04-24T23:16:14.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/power_domain.txt 2024-04-24T23:16:14.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/renesas,sysc-rmobile.txt 2024-04-24T23:16:14.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ 2024-04-24T23:16:14.856 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/keystone-reset.txt 2024-04-24T23:16:14.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/ltc2952-poweroff.txt 2024-04-24T23:16:14.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/st-reset.txt 2024-04-24T23:16:14.859 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/reset/syscon-reboot.txt 2024-04-24T23:16:14.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rockchip-io-domain.txt 2024-04-24T23:16:14.861 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/rx51-battery.txt 2024-04-24T23:16:14.862 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power/twl-charger.txt 2024-04-24T23:16:14.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ 2024-04-24T23:16:14.864 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/ 2024-04-24T23:16:14.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/btemp.txt 2024-04-24T23:16:14.866 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/chargalg.txt 2024-04-24T23:16:14.867 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/charger.txt 2024-04-24T23:16:14.868 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ab8500/fg.txt 2024-04-24T23:16:14.870 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/axxia-reset.txt 2024-04-24T23:16:14.871 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/charger-manager.txt 2024-04-24T23:16:14.872 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/gpio-charger.txt 2024-04-24T23:16:14.873 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/imx-snvs-poweroff.txt 2024-04-24T23:16:14.874 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/lp8727_charger.txt 2024-04-24T23:16:14.875 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max17042_battery.txt 2024-04-24T23:16:14.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/max8925_batter.txt 2024-04-24T23:16:14.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/msm-poweroff.txt 2024-04-24T23:16:14.878 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/olpc_battery.txt 2024-04-24T23:16:14.879 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/power_supply.txt 2024-04-24T23:16:14.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/qnap-poweroff.txt 2024-04-24T23:16:14.881 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/restart-poweroff.txt 2024-04-24T23:16:14.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/sbs_sbs-battery.txt 2024-04-24T23:16:14.883 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/ti,bq24735.txt 2024-04-24T23:16:14.884 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/power_supply/tps65090.txt 2024-04-24T23:16:14.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/ 2024-04-24T23:16:14.886 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ 2024-04-24T23:16:14.887 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/akebono.txt 2024-04-24T23:16:14.888 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/cpm.txt 2024-04-24T23:16:14.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/emac.txt 2024-04-24T23:16:14.890 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/hsta.txt 2024-04-24T23:16:14.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ndfc.txt 2024-04-24T23:16:14.892 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/ppc440spe-adma.txt 2024-04-24T23:16:14.893 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/4xx/reboot.txt 2024-04-24T23:16:14.894 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ 2024-04-24T23:16:14.895 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/board.txt 2024-04-24T23:16:14.896 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cache_sram.txt 2024-04-24T23:16:14.897 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ccf.txt 2024-04-24T23:16:14.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/ 2024-04-24T23:16:14.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm.txt 2024-04-24T23:16:14.900 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/ 2024-04-24T23:16:14.901 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/brg.txt 2024-04-24T23:16:14.902 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/i2c.txt 2024-04-24T23:16:14.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/pic.txt 2024-04-24T23:16:14.904 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/cpm/usb.txt 2024-04-24T23:16:14.905 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/gpio.txt 2024-04-24T23:16:14.906 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/network.txt 2024-04-24T23:16:14.907 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe.txt 2024-04-24T23:16:14.908 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ 2024-04-24T23:16:14.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/firmware.txt 2024-04-24T23:16:14.910 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/par_io.txt 2024-04-24T23:16:14.911 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/pincfg.txt 2024-04-24T23:16:14.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/ucc.txt 2024-04-24T23:16:14.913 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/qe/usb.txt 2024-04-24T23:16:14.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpm_qe/serial.txt 2024-04-24T23:16:14.915 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/cpus.txt 2024-04-24T23:16:14.916 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dcsr.txt 2024-04-24T23:16:15.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/diu.txt 2024-04-24T23:16:15.170 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/dma.txt 2024-04-24T23:16:15.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/ecm.txt 2024-04-24T23:16:15.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/fman.txt 2024-04-24T23:16:15.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/gtm.txt 2024-04-24T23:16:15.174 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/guts.txt 2024-04-24T23:16:15.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/interlaken-lac.txt 2024-04-24T23:16:15.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/l2cache.txt 2024-04-24T23:16:15.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/lbc.txt 2024-04-24T23:16:15.178 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcm.txt 2024-04-24T23:16:15.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mcu-mpc8349emitx.txt 2024-04-24T23:16:15.180 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mem-ctrlr.txt 2024-04-24T23:16:15.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5121-psc.txt 2024-04-24T23:16:15.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpc5200.txt 2024-04-24T23:16:15.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-msgr.txt 2024-04-24T23:16:15.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic-timer.txt 2024-04-24T23:16:15.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/mpic.txt 2024-04-24T23:16:15.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/msi-pic.txt 2024-04-24T23:16:15.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pamu.txt 2024-04-24T23:16:15.189 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/pmc.txt 2024-04-24T23:16:15.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/raideng.txt 2024-04-24T23:16:15.191 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio-rmu.txt 2024-04-24T23:16:15.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/fsl/srio.txt 2024-04-24T23:16:15.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/ 2024-04-24T23:16:15.194 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/gamecube.txt 2024-04-24T23:16:15.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/powerpc/nintendo/wii.txt 2024-04-24T23:16:15.196 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/ 2024-04-24T23:16:15.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pps/pps-gpio.txt 2024-04-24T23:16:15.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ 2024-04-24T23:16:15.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-hlcdc-pwm.txt 2024-04-24T23:16:15.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-pwm.txt 2024-04-24T23:16:15.200 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/atmel-tcb-pwm.txt 2024-04-24T23:16:15.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/bcm-kona-pwm.txt 2024-04-24T23:16:15.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/cirrus,clps711x-pwm.txt 2024-04-24T23:16:15.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/img-pwm.txt 2024-04-24T23:16:15.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/imx-pwm.txt 2024-04-24T23:16:15.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/lpc32xx-pwm.txt 2024-04-24T23:16:15.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/mxs-pwm.txt 2024-04-24T23:16:15.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nvidia,tegra20-pwm.txt 2024-04-24T23:16:15.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/nxp,pca9685-pwm.txt 2024-04-24T23:16:15.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-bcm2835.txt 2024-04-24T23:16:15.210 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-fsl-ftm.txt 2024-04-24T23:16:15.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-lp3943.txt 2024-04-24T23:16:15.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-rockchip.txt 2024-04-24T23:16:15.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-samsung.txt 2024-04-24T23:16:15.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-st.txt 2024-04-24T23:16:15.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-sun4i.txt 2024-04-24T23:16:15.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiecap.txt 2024-04-24T23:16:15.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tiehrpwm.txt 2024-04-24T23:16:15.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm-tipwmss.txt 2024-04-24T23:16:15.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pwm.txt 2024-04-24T23:16:15.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/pxa-pwm.txt 2024-04-24T23:16:15.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/renesas,tpu-pwm.txt 2024-04-24T23:16:15.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/spear-pwm.txt 2024-04-24T23:16:15.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwm.txt 2024-04-24T23:16:15.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/ti,twl-pwmled.txt 2024-04-24T23:16:15.225 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/pwm/vt8500-pwm.txt 2024-04-24T23:16:15.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/ 2024-04-24T23:16:15.227 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regmap/regmap.txt 2024-04-24T23:16:15.228 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ 2024-04-24T23:16:15.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm800.txt 2024-04-24T23:16:15.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/88pm860x.txt 2024-04-24T23:16:15.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/act8865-regulator.txt 2024-04-24T23:16:15.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/anatop-regulator.txt 2024-04-24T23:16:15.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/as3722-regulator.txt 2024-04-24T23:16:15.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9210.txt 2024-04-24T23:16:15.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/da9211.txt 2024-04-24T23:16:15.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fan53555.txt 2024-04-24T23:16:15.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/fixed-regulator.txt 2024-04-24T23:16:15.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/gpio-regulator.txt 2024-04-24T23:16:15.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/isl9305.txt 2024-04-24T23:16:15.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/lp872x.txt 2024-04-24T23:16:15.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ltc3589.txt 2024-04-24T23:16:15.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max1586-regulator.txt 2024-04-24T23:16:15.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max77802.txt 2024-04-24T23:16:15.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8660.txt 2024-04-24T23:16:15.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8907.txt 2024-04-24T23:16:15.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8925-regulator.txt 2024-04-24T23:16:15.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8952.txt 2024-04-24T23:16:15.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8973-regulator.txt 2024-04-24T23:16:15.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/max8997-regulator.txt 2024-04-24T23:16:15.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/mt6397-regulator.txt 2024-04-24T23:16:15.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/palmas-pmic.txt 2024-04-24T23:16:15.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pbias-regulator.txt 2024-04-24T23:16:15.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pfuze100.txt 2024-04-24T23:16:15.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/pwm-regulator.txt 2024-04-24T23:16:15.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/regulator.txt 2024-04-24T23:16:15.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/s5m8767-regulator.txt 2024-04-24T23:16:15.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/sky81452-regulator.txt 2024-04-24T23:16:15.256 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/ti-abb-regulator.txt 2024-04-24T23:16:15.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps51632-regulator.txt 2024-04-24T23:16:15.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps62360-regulator.txt 2024-04-24T23:16:15.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65090.txt 2024-04-24T23:16:15.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65217.txt 2024-04-24T23:16:15.260 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps65218.txt 2024-04-24T23:16:15.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/tps6586x.txt 2024-04-24T23:16:15.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/twl-regulator.txt 2024-04-24T23:16:15.263 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/regulator/vexpress.txt 2024-04-24T23:16:15.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/ 2024-04-24T23:16:15.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt 2024-04-24T23:16:15.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/ 2024-04-24T23:16:15.266 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/allwinner,sunxi-clock-reset.txt 2024-04-24T23:16:15.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/fsl,imx-src.txt 2024-04-24T23:16:15.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/reset.txt 2024-04-24T23:16:15.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/sirf,rstc.txt 2024-04-24T23:16:15.270 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/socfpga-reset.txt 2024-04-24T23:16:15.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-picophyreset.txt 2024-04-24T23:16:15.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-powerdown.txt 2024-04-24T23:16:15.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/reset/st,sti-softreset.txt 2024-04-24T23:16:15.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/resource-names.txt 2024-04-24T23:16:15.275 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/ 2024-04-24T23:16:15.276 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/apm,rng.txt 2024-04-24T23:16:15.277 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/brcm,bcm2835.txt 2024-04-24T23:16:15.278 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rng/qcom,prng.txt 2024-04-24T23:16:15.279 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/ 2024-04-24T23:16:15.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/armada-380-rtc.txt 2024-04-24T23:16:15.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91rm9200-rtc.txt 2024-04-24T23:16:15.281 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/atmel,at91sam9-rtc.txt 2024-04-24T23:16:15.282 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dallas,ds1339.txt 2024-04-24T23:16:15.283 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/dw-apb.txt 2024-04-24T23:16:15.284 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/haoyu,hym8563.txt 2024-04-24T23:16:15.284 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/imxdi-rtc.txt 2024-04-24T23:16:15.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/isil,isl12057.txt 2024-04-24T23:16:15.286 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/lpc32xx-rtc.txt 2024-04-24T23:16:15.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/maxim,ds1742.txt 2024-04-24T23:16:15.288 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/moxa,moxart-rtc.txt 2024-04-24T23:16:15.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nvidia,tegra20-rtc.txt 2024-04-24T23:16:15.290 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/nxp,rtc-2123.txt 2024-04-24T23:16:15.291 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/olpc-xo1-rtc.txt 2024-04-24T23:16:15.292 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/orion-rtc.txt 2024-04-24T23:16:15.293 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/pxa-rtc.txt 2024-04-24T23:16:15.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-cmos.txt 2024-04-24T23:16:15.295 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-omap.txt 2024-04-24T23:16:15.296 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-opal.txt 2024-04-24T23:16:15.297 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/rtc-palmas.txt 2024-04-24T23:16:15.298 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/s3c-rtc.txt 2024-04-24T23:16:15.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sa1100-rtc.txt 2024-04-24T23:16:15.300 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/snvs-rtc.txt 2024-04-24T23:16:15.301 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/spear-rtc.txt 2024-04-24T23:16:15.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/stmp3xxx-rtc.txt 2024-04-24T23:16:15.303 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sun6i-rtc.txt 2024-04-24T23:16:15.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/sunxi-rtc.txt 2024-04-24T23:16:15.305 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/twl-rtc.txt 2024-04-24T23:16:15.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/via,vt8500-rtc.txt 2024-04-24T23:16:15.307 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/rtc/xgene-rtc.txt 2024-04-24T23:16:15.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/ 2024-04-24T23:16:15.309 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/ 2024-04-24T23:16:15.310 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/security/tpm/st33zp24-i2c.txt 2024-04-24T23:16:15.311 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/ 2024-04-24T23:16:15.312 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/8250.txt 2024-04-24T23:16:15.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_jtaguart.txt 2024-04-24T23:16:15.314 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/altera_uart.txt 2024-04-24T23:16:15.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/arc-uart.txt 2024-04-24T23:16:15.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/atmel-usart.txt 2024-04-24T23:16:15.317 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/axis,etraxfs-uart.txt 2024-04-24T23:16:15.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/bcm63xx-uart.txt 2024-04-24T23:16:15.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cavium-uart.txt 2024-04-24T23:16:15.319 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cdns,uart.txt 2024-04-24T23:16:15.320 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/cirrus,clps711x-uart.txt 2024-04-24T23:16:15.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/digicolor-usart.txt 2024-04-24T23:16:15.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/efm32-uart.txt 2024-04-24T23:16:15.323 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-imx-uart.txt 2024-04-24T23:16:15.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-lpuart.txt 2024-04-24T23:16:15.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/fsl-mxs-auart.txt 2024-04-24T23:16:15.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/lantiq_asc.txt 2024-04-24T23:16:15.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/maxim,max310x.txt 2024-04-24T23:16:15.327 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl,pxa-ssp.txt 2024-04-24T23:16:15.328 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mrvl-serial.txt 2024-04-24T23:16:15.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/mtk-uart.txt 2024-04-24T23:16:15.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nvidia,tegra20-hsuart.txt 2024-04-24T23:16:15.331 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp,sc16is7xx.txt 2024-04-24T23:16:15.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/nxp-lpc32xx-hsuart.txt 2024-04-24T23:16:15.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/omap_serial.txt 2024-04-24T23:16:15.334 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/pl011.txt 2024-04-24T23:16:15.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qca,ar9330-uart.txt 2024-04-24T23:16:15.336 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uart.txt 2024-04-24T23:16:15.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/qcom,msm-uartdm.txt 2024-04-24T23:16:15.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/renesas,sci-serial.txt 2024-04-24T23:16:15.339 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/rs485.txt 2024-04-24T23:16:15.340 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/samsung_uart.txt 2024-04-24T23:16:15.341 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sirf-uart.txt 2024-04-24T23:16:15.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/snps-dw-apb-uart.txt 2024-04-24T23:16:15.343 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/sprd-uart.txt 2024-04-24T23:16:15.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/st-asc.txt 2024-04-24T23:16:15.345 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serial/vt8500-uart.txt 2024-04-24T23:16:15.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/ 2024-04-24T23:16:15.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/allwinner,sun4i-ps2.txt 2024-04-24T23:16:15.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/altera_ps2.txt 2024-04-24T23:16:15.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/olpc,ap-sp.txt 2024-04-24T23:16:15.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/serio/snps-arc_ps2.txt 2024-04-24T23:16:15.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ 2024-04-24T23:16:15.352 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/ 2024-04-24T23:16:15.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman-portals.txt 2024-04-24T23:16:15.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/bman.txt 2024-04-24T23:16:15.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman-portals.txt 2024-04-24T23:16:15.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/fsl/qman.txt 2024-04-24T23:16:15.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/ 2024-04-24T23:16:15.358 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/qcom/qcom,gsbi.txt 2024-04-24T23:16:15.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/ 2024-04-24T23:16:15.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-dma.txt 2024-04-24T23:16:15.361 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/soc/ti/keystone-navigator-qmss.txt 2024-04-24T23:16:15.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ 2024-04-24T23:16:15.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,adau1701.txt 2024-04-24T23:16:15.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-i2s.txt 2024-04-24T23:16:15.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,axi-spdif-tx.txt 2024-04-24T23:16:15.366 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/adi,ssm2602.txt 2024-04-24T23:16:15.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4104.txt 2024-04-24T23:16:15.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4554.c 2024-04-24T23:16:15.369 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak4642.txt 2024-04-24T23:16:15.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ak5386.txt 2024-04-24T23:16:15.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5623.txt 2024-04-24T23:16:15.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/alc5632.txt 2024-04-24T23:16:15.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/armada-370db-audio.txt 2024-04-24T23:16:15.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/arndale.txt 2024-04-24T23:16:15.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-at91sam9g20ek-wm8731-audio.txt 2024-04-24T23:16:15.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-sam9x5-wm8731-audio.txt 2024-04-24T23:16:15.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel-wm8904.txt 2024-04-24T23:16:15.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/atmel_ac97c.txt 2024-04-24T23:16:15.379 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/bcm2835-i2s.txt 2024-04-24T23:16:15.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cdns,xtfpga-i2s.txt 2024-04-24T23:16:15.381 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs35l32.txt 2024-04-24T23:16:15.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4265.txt 2024-04-24T23:16:15.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4270.txt 2024-04-24T23:16:15.383 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs4271.txt 2024-04-24T23:16:15.384 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l52.txt 2024-04-24T23:16:15.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l56.txt 2024-04-24T23:16:15.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42l73.txt 2024-04-24T23:16:15.387 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/cs42xx8.txt 2024-04-24T23:16:15.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/da9055.txt 2024-04-24T23:16:15.389 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-evm-audio.txt 2024-04-24T23:16:15.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/davinci-mcasp-audio.txt 2024-04-24T23:16:15.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/designware-i2s.txt 2024-04-24T23:16:15.392 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/es8328.txt 2024-04-24T23:16:15.392 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/eukrea-tlv320.txt 2024-04-24T23:16:15.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,asrc.txt 2024-04-24T23:16:15.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,esai.txt 2024-04-24T23:16:15.395 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,spdif.txt 2024-04-24T23:16:15.396 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl,ssi.txt 2024-04-24T23:16:15.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-asoc-card.txt 2024-04-24T23:16:15.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/fsl-sai.txt 2024-04-24T23:16:15.399 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/hdmi.txt 2024-04-24T23:16:15.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-es8328.txt 2024-04-24T23:16:15.401 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-sgtl5000.txt 2024-04-24T23:16:15.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-spdif.txt 2024-04-24T23:16:15.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audio-wm8962.txt 2024-04-24T23:16:15.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/imx-audmux.txt 2024-04-24T23:16:15.405 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ingenic,jz4740-i2s.txt 2024-04-24T23:16:15.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98090.txt 2024-04-24T23:16:15.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98095.txt 2024-04-24T23:16:15.408 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/max98357a.txt 2024-04-24T23:16:15.409 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa-ssp.txt 2024-04-24T23:16:15.410 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mrvl,pxa2xx-pcm.txt 2024-04-24T23:16:15.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mvebu-audio.txt 2024-04-24T23:16:15.412 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-audio-sgtl5000.txt 2024-04-24T23:16:15.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/mxs-saif.txt 2024-04-24T23:16:15.414 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nokia,rx51.txt 2024-04-24T23:16:15.415 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-alc5632.txt 2024-04-24T23:16:15.416 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-max98090.txt 2024-04-24T23:16:15.417 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5640.txt 2024-04-24T23:16:15.418 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-rt5677.txt 2024-04-24T23:16:15.419 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-trimslice.txt 2024-04-24T23:16:15.420 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8753.txt 2024-04-24T23:16:15.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm8903.txt 2024-04-24T23:16:15.422 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra-audio-wm9712.txt 2024-04-24T23:16:15.423 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-ac97.txt 2024-04-24T23:16:15.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-das.txt 2024-04-24T23:16:15.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra20-i2s.txt 2024-04-24T23:16:15.425 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-ahub.txt 2024-04-24T23:16:15.426 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-hda.txt 2024-04-24T23:16:15.427 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/nvidia,tegra30-i2s.txt 2024-04-24T23:16:15.428 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-abe-twl6040.txt 2024-04-24T23:16:15.429 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-dmic.txt 2024-04-24T23:16:15.430 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcbsp.txt 2024-04-24T23:16:15.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-mcpdm.txt 2024-04-24T23:16:15.432 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/omap-twl4030.txt 2024-04-24T23:16:15.433 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm1792a.txt 2024-04-24T23:16:15.434 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/pcm512x.txt 2024-04-24T23:16:15.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,fsi.txt 2024-04-24T23:16:15.436 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/renesas,rsnd.txt 2024-04-24T23:16:15.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rockchip-i2s.txt 2024-04-24T23:16:15.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5631.txt 2024-04-24T23:16:15.438 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5640.txt 2024-04-24T23:16:15.439 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/rt5677.txt 2024-04-24T23:16:15.440 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,odroidx2-max98090.txt 2024-04-24T23:16:15.441 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung,smdk-wm8994.txt 2024-04-24T23:16:15.442 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/samsung-i2s.txt 2024-04-24T23:16:15.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sgtl5000.txt 2024-04-24T23:16:15.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/simple-card.txt 2024-04-24T23:16:15.444 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-codec.txt 2024-04-24T23:16:15.445 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio-port.txt 2024-04-24T23:16:15.446 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-audio.txt 2024-04-24T23:16:15.447 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/sirf-usp.txt 2024-04-24T23:16:15.448 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/snow.txt 2024-04-24T23:16:15.449 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/soc-ac97link.txt 2024-04-24T23:16:15.450 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-receiver.txt 2024-04-24T23:16:15.451 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/spdif-transmitter.txt 2024-04-24T23:16:15.452 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm2518.txt 2024-04-24T23:16:15.453 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ssm4567.txt 2024-04-24T23:16:15.454 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta32x.txt 2024-04-24T23:16:15.455 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/st,sta350.txt 2024-04-24T23:16:15.456 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tas2552.txt 2024-04-24T23:16:15.457 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tdm-slot.txt 2024-04-24T23:16:15.458 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,pcm1681.txt 2024-04-24T23:16:15.459 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ti,tas5086.txt 2024-04-24T23:16:15.460 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic31xx.txt 2024-04-24T23:16:15.461 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic32x4.txt 2024-04-24T23:16:15.462 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tlv320aic3x.txt 2024-04-24T23:16:15.463 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/tpa6130a2.txt 2024-04-24T23:16:15.464 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ts3a227e.txt 2024-04-24T23:16:15.465 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-mop500.txt 2024-04-24T23:16:15.466 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/ux500-msp.txt 2024-04-24T23:16:15.467 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/widgets.txt 2024-04-24T23:16:15.468 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8510.txt 2024-04-24T23:16:15.469 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8523.txt 2024-04-24T23:16:15.470 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8580.txt 2024-04-24T23:16:15.471 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8711.txt 2024-04-24T23:16:15.472 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8728.txt 2024-04-24T23:16:15.473 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8731.txt 2024-04-24T23:16:15.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8737.txt 2024-04-24T23:16:15.475 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8741.txt 2024-04-24T23:16:15.476 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8750.txt 2024-04-24T23:16:15.477 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8753.txt 2024-04-24T23:16:15.478 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8770.txt 2024-04-24T23:16:15.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8776.txt 2024-04-24T23:16:15.480 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8804.txt 2024-04-24T23:16:15.481 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8903.txt 2024-04-24T23:16:15.482 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8904.txt 2024-04-24T23:16:15.483 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8960.txt 2024-04-24T23:16:15.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8962.txt 2024-04-24T23:16:15.485 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/sound/wm8994.txt 2024-04-24T23:16:15.486 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ 2024-04-24T23:16:15.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/brcm,bcm2835-spi.txt 2024-04-24T23:16:15.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/efm32-spi.txt 2024-04-24T23:16:15.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-imx-cspi.txt 2024-04-24T23:16:15.491 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/fsl-spi.txt 2024-04-24T23:16:15.492 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/mxs-spi.txt 2024-04-24T23:16:15.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra114-spi.txt 2024-04-24T23:16:15.494 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-sflash.txt 2024-04-24T23:16:15.495 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/nvidia,tegra20-slink.txt 2024-04-24T23:16:15.496 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/omap-spi.txt 2024-04-24T23:16:15.497 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/qcom,spi-qup.txt 2024-04-24T23:16:15.498 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-hspi.txt 2024-04-24T23:16:15.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/sh-msiof.txt 2024-04-24T23:16:15.500 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/snps,dw-apb-ssi.txt 2024-04-24T23:16:15.501 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-bus.txt 2024-04-24T23:16:15.502 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-cadence.txt 2024-04-24T23:16:15.503 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-davinci.txt 2024-04-24T23:16:15.504 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-dw.txt 2024-04-24T23:16:15.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-fsl-dspi.txt 2024-04-24T23:16:15.506 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-gpio.txt 2024-04-24T23:16:15.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-img-spfi.txt 2024-04-24T23:16:15.508 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-meson.txt 2024-04-24T23:16:15.509 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-octeon.txt 2024-04-24T23:16:15.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-orion.txt 2024-04-24T23:16:15.511 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rockchip.txt 2024-04-24T23:16:15.512 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-rspi.txt 2024-04-24T23:16:15.513 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-samsung.txt 2024-04-24T23:16:15.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sc18is602.txt 2024-04-24T23:16:15.515 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sirf.txt 2024-04-24T23:16:15.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-st-ssc.txt 2024-04-24T23:16:15.517 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun4i.txt 2024-04-24T23:16:15.518 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-sun6i.txt 2024-04-24T23:16:15.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi-xtensa-xtfpga.txt 2024-04-24T23:16:15.520 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_altera.txt 2024-04-24T23:16:15.521 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_atmel.txt 2024-04-24T23:16:15.522 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_oc_tiny.txt 2024-04-24T23:16:15.523 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/spi_pl022.txt 2024-04-24T23:16:15.524 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spi/ti_qspi.txt 2024-04-24T23:16:15.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/ 2024-04-24T23:16:15.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/qcom,spmi-pmic-arb.txt 2024-04-24T23:16:15.527 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/spmi/spmi.txt 2024-04-24T23:16:15.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/ 2024-04-24T23:16:15.529 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/ 2024-04-24T23:16:15.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/ 2024-04-24T23:16:15.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/lpc32xx-adc.txt 2024-04-24T23:16:15.532 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/mxs-lradc.txt 2024-04-24T23:16:15.533 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/staging/iio/adc/spear-adc.txt 2024-04-24T23:16:15.534 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/submitting-patches.txt 2024-04-24T23:16:15.535 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ 2024-04-24T23:16:15.536 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/armada-thermal.txt 2024-04-24T23:16:15.537 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/db8500-thermal.txt 2024-04-24T23:16:15.538 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/dove-thermal.txt 2024-04-24T23:16:15.539 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/exynos-thermal.txt 2024-04-24T23:16:15.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/imx-thermal.txt 2024-04-24T23:16:15.541 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/kirkwood-thermal.txt 2024-04-24T23:16:15.542 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rcar-thermal.txt 2024-04-24T23:16:15.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/rockchip-thermal.txt 2024-04-24T23:16:15.544 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/spear-thermal.txt 2024-04-24T23:16:15.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/st-thermal.txt 2024-04-24T23:16:15.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/tegra-soctherm.txt 2024-04-24T23:16:15.547 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/thermal.txt 2024-04-24T23:16:15.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/thermal/ti_soc_thermal.txt 2024-04-24T23:16:15.549 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ 2024-04-24T23:16:15.550 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun4i-timer.txt 2024-04-24T23:16:15.551 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/allwinner,sun5i-a13-hstimer.txt 2024-04-24T23:16:15.552 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/amlogic,meson6-timer.txt 2024-04-24T23:16:15.553 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/arm,sp804.txt 2024-04-24T23:16:15.554 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/brcm,bcm2835-system-timer.txt 2024-04-24T23:16:15.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cadence,ttc-timer.txt 2024-04-24T23:16:15.556 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/cirrus,clps711x-timer.txt 2024-04-24T23:16:15.557 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/digicolor-timer.txt 2024-04-24T23:16:15.558 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/energymicro,efm32-timer.txt 2024-04-24T23:16:15.559 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,ftm-timer.txt 2024-04-24T23:16:15.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/fsl,imxgpt.txt 2024-04-24T23:16:15.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/lsi,zevio-timer.txt 2024-04-24T23:16:15.562 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,armada-370-xp-timer.txt 2024-04-24T23:16:15.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/marvell,orion-timer.txt 2024-04-24T23:16:15.564 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/mediatek,mtk-timer.txt 2024-04-24T23:16:15.565 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/moxa,moxart-timer.txt 2024-04-24T23:16:15.566 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra20-timer.txt 2024-04-24T23:16:15.568 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/nvidia,tegra30-timer.txt 2024-04-24T23:16:15.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,cmt.txt 2024-04-24T23:16:15.570 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,mtu2.txt 2024-04-24T23:16:15.571 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/renesas,tmu.txt 2024-04-24T23:16:15.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/rockchip,rk3288-timer.txt 2024-04-24T23:16:15.573 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/samsung,exynos4210-mct.txt 2024-04-24T23:16:15.575 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/stericsson-u300-apptimer.txt 2024-04-24T23:16:15.575 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/timer/ti,keystone-timer.txt 2024-04-24T23:16:15.576 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ 2024-04-24T23:16:15.577 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/ufs/ufshcd-pltfrm.txt 2024-04-24T23:16:15.578 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/unittest.txt 2024-04-24T23:16:15.579 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ 2024-04-24T23:16:15.580 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/am33xx-usb.txt 2024-04-24T23:16:15.581 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/atmel-usb.txt 2024-04-24T23:16:15.582 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-imx.txt 2024-04-24T23:16:15.583 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-qcom.txt 2024-04-24T23:16:15.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-usb2.txt 2024-04-24T23:16:15.585 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ci-hdrc-zevio.txt 2024-04-24T23:16:15.586 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc2.txt 2024-04-24T23:16:15.587 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3-st.txt 2024-04-24T23:16:15.588 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/dwc3.txt 2024-04-24T23:16:15.589 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-omap.txt 2024-04-24T23:16:15.590 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-orion.txt 2024-04-24T23:16:15.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ehci-st.txt 2024-04-24T23:16:15.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/exynos-usb.txt 2024-04-24T23:16:15.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/fsl-usb.txt 2024-04-24T23:16:15.594 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/generic.txt 2024-04-24T23:16:15.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/gr-udc.txt 2024-04-24T23:16:15.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/isp1301.txt 2024-04-24T23:16:15.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-phy.txt 2024-04-24T23:16:15.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/keystone-usb.txt 2024-04-24T23:16:15.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/lpc32xx-udc.txt 2024-04-24T23:16:15.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/msm-hsusb.txt 2024-04-24T23:16:15.601 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/mxs-phy.txt 2024-04-24T23:16:15.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-ehci.txt 2024-04-24T23:16:15.603 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/nvidia,tegra20-usb-phy.txt 2024-04-24T23:16:15.604 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-nxp.txt 2024-04-24T23:16:15.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-omap3.txt 2024-04-24T23:16:15.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ohci-st.txt 2024-04-24T23:16:15.607 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/omap-usb.txt 2024-04-24T23:16:15.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/pxa-usb.txt 2024-04-24T23:16:15.609 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/qcom,dwc3.txt 2024-04-24T23:16:15.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/renesas_usbhs.txt 2024-04-24T23:16:15.611 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-hsotg.txt 2024-04-24T23:16:15.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/samsung-usbphy.txt 2024-04-24T23:16:15.613 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/spear-usb.txt 2024-04-24T23:16:15.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/twlxxxx-usb.txt 2024-04-24T23:16:15.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/udc-xilinx.txt 2024-04-24T23:16:15.616 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ehci.txt 2024-04-24T23:16:15.617 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-nop-xceiv.txt 2024-04-24T23:16:15.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-ohci.txt 2024-04-24T23:16:15.619 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-uhci.txt 2024-04-24T23:16:15.620 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb-xhci.txt 2024-04-24T23:16:15.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usb3503.txt 2024-04-24T23:16:15.622 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/usbmisc-imx.txt 2024-04-24T23:16:15.623 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/usb/ux500-usb.txt 2024-04-24T23:16:15.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/vendor-prefixes.txt 2024-04-24T23:16:15.625 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ 2024-04-24T23:16:15.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7123.txt 2024-04-24T23:16:15.627 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/adi,adv7511.txt 2024-04-24T23:16:15.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/analog-tv-connector.txt 2024-04-24T23:16:15.629 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/arm,pl11x.txt 2024-04-24T23:16:15.630 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/atmel,lcdc.txt 2024-04-24T23:16:15.631 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/ 2024-04-24T23:16:15.632 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/88pm860x.txt 2024-04-24T23:16:15.633 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/gpio-backlight.txt 2024-04-24T23:16:15.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/lp855x.txt 2024-04-24T23:16:15.635 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/max8925-backlight.txt 2024-04-24T23:16:15.636 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/pwm-backlight.txt 2024-04-24T23:16:15.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/backlight/tps65217-backlight.txt 2024-04-24T23:16:15.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ 2024-04-24T23:16:15.639 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ps8622.txt 2024-04-24T23:16:15.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/bridge/ptn3460.txt 2024-04-24T23:16:15.641 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/cirrus,clps711x-fb.txt 2024-04-24T23:16:15.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/display-timing.txt 2024-04-24T23:16:15.643 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dvi-connector.txt 2024-04-24T23:16:15.644 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/dw_hdmi-rockchip.txt 2024-04-24T23:16:15.645 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos7-decon.txt 2024-04-24T23:16:15.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dp.txt 2024-04-24T23:16:15.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_dsim.txt 2024-04-24T23:16:15.648 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmi.txt 2024-04-24T23:16:15.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiddc.txt 2024-04-24T23:16:15.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_hdmiphy.txt 2024-04-24T23:16:15.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/exynos_mixer.txt 2024-04-24T23:16:15.652 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/fsl,imx-fb.txt 2024-04-24T23:16:15.653 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/hdmi-connector.txt 2024-04-24T23:16:15.654 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/lgphilips,lb035q02.txt 2024-04-24T23:16:15.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dpi.txt 2024-04-24T23:16:15.656 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/panel-dsi-cm.txt 2024-04-24T23:16:15.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/renesas,du.txt 2024-04-24T23:16:15.658 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-drm.txt 2024-04-24T23:16:15.659 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/rockchip-vop.txt 2024-04-24T23:16:15.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/samsung-fimd.txt 2024-04-24T23:16:15.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sharp,ls037v7dw01.txt 2024-04-24T23:16:15.879 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer-sunxi.txt 2024-04-24T23:16:15.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/simple-framebuffer.txt 2024-04-24T23:16:15.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/sony,acx565akm.txt 2024-04-24T23:16:15.883 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1289fb.txt 2024-04-24T23:16:15.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ssd1307fb.txt 2024-04-24T23:16:15.887 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/thine,thc63lvdm83d 2024-04-24T23:16:15.888 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,dra7-dss.txt 2024-04-24T23:16:15.890 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap-dss.txt 2024-04-24T23:16:15.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap2-dss.txt 2024-04-24T23:16:15.893 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap3-dss.txt 2024-04-24T23:16:15.895 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap4-dss.txt 2024-04-24T23:16:15.896 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,omap5-dss.txt 2024-04-24T23:16:15.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,opa362.txt 2024-04-24T23:16:15.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tfp410.txt 2024-04-24T23:16:15.901 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/ti,tpd12s015.txt 2024-04-24T23:16:15.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/toppoly,td028ttec1.txt 2024-04-24T23:16:15.904 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/tpo,td043mtea1.txt 2024-04-24T23:16:15.906 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/vga-connector.txt 2024-04-24T23:16:15.907 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/via,vt8500-fb.txt 2024-04-24T23:16:15.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,prizm-ge-rops.txt 2024-04-24T23:16:15.911 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/video/wm,wm8505-fb.txt 2024-04-24T23:16:15.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/ 2024-04-24T23:16:15.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/virtio/mmio.txt 2024-04-24T23:16:15.915 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/ 2024-04-24T23:16:15.917 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/fsl-imx-owire.txt 2024-04-24T23:16:15.918 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/omap-hdq.txt 2024-04-24T23:16:15.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/w1/w1-gpio.txt 2024-04-24T23:16:15.921 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ 2024-04-24T23:16:15.923 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-at91rm9200-wdt.txt 2024-04-24T23:16:15.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/atmel-wdt.txt 2024-04-24T23:16:15.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/brcm,bcm2835-pm-wdog.txt 2024-04-24T23:16:15.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/cadence-wdt.txt 2024-04-24T23:16:15.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/davinci-wdt.txt 2024-04-24T23:16:15.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/dw_wdt.txt 2024-04-24T23:16:15.932 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/fsl-imx-wdt.txt 2024-04-24T23:16:15.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt 2024-04-24T23:16:15.934 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/imgpdc-wdt.txt 2024-04-24T23:16:15.936 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/ingenic,jz4740-wdt.txt 2024-04-24T23:16:15.938 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/marvel.txt 2024-04-24T23:16:15.939 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/men-a021-wdt.txt 2024-04-24T23:16:15.941 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/meson6-wdt.txt 2024-04-24T23:16:15.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/moxa,moxart-watchdog.txt 2024-04-24T23:16:15.944 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/mtk-wdt.txt 2024-04-24T23:16:15.945 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/of-xilinx-wdt.txt 2024-04-24T23:16:15.947 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/omap-wdt.txt 2024-04-24T23:16:15.948 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/pnx4008-wdt.txt 2024-04-24T23:16:15.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qca-ar7130-wdt.txt 2024-04-24T23:16:15.951 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/qcom-wdt.txt 2024-04-24T23:16:15.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/rt2880-wdt.txt 2024-04-24T23:16:15.954 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/samsung-wdt.txt 2024-04-24T23:16:15.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sirfsoc_wdt.txt 2024-04-24T23:16:15.957 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/stericsson-coh901327.txt 2024-04-24T23:16:15.959 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/sunxi-wdt.txt 2024-04-24T23:16:15.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/watchdog/twl4030-wdt.txt 2024-04-24T23:16:15.962 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ 2024-04-24T23:16:15.963 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/ce4100.txt 2024-04-24T23:16:15.964 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/interrupt.txt 2024-04-24T23:16:15.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/x86/timer.txt 2024-04-24T23:16:15.966 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/xilinx.txt 2024-04-24T23:16:15.967 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/ 2024-04-24T23:16:15.968 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/bindings/xillybus/xillybus.txt 2024-04-24T23:16:15.969 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/booting-without-of.txt 2024-04-24T23:16:15.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/changesets.txt 2024-04-24T23:16:15.971 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/dynamic-resolution-notes.txt 2024-04-24T23:16:15.972 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/of_selftest.txt 2024-04-24T23:16:15.973 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/overlay-notes.txt 2024-04-24T23:16:15.974 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/todo.txt 2024-04-24T23:16:15.975 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/devicetree/usage-model.txt 2024-04-24T23:16:15.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/digsig.txt 2024-04-24T23:16:15.977 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dma-buf-sharing.txt 2024-04-24T23:16:15.978 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/ 2024-04-24T23:16:15.979 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/00-INDEX 2024-04-24T23:16:15.980 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/client.txt 2024-04-24T23:16:15.981 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/dmatest.txt 2024-04-24T23:16:15.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dmaengine/provider.txt 2024-04-24T23:16:15.983 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dontdiff 2024-04-24T23:16:15.984 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/ 2024-04-24T23:16:15.985 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/binding.txt 2024-04-24T23:16:15.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/bus.txt 2024-04-24T23:16:15.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/class.txt 2024-04-24T23:16:15.987 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/design-patterns.txt 2024-04-24T23:16:15.988 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/device.txt 2024-04-24T23:16:15.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/devres.txt 2024-04-24T23:16:15.990 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/driver.txt 2024-04-24T23:16:15.991 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/overview.txt 2024-04-24T23:16:15.992 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/platform.txt 2024-04-24T23:16:15.993 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/driver-model/porting.txt 2024-04-24T23:16:15.994 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/ 2024-04-24T23:16:15.995 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/README.dvb-usb 2024-04-24T23:16:15.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/avermedia.txt 2024-04-24T23:16:15.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/bt8xx.txt 2024-04-24T23:16:15.997 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/cards.txt 2024-04-24T23:16:15.998 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/ci.txt 2024-04-24T23:16:15.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/contributors.txt 2024-04-24T23:16:16.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/faq.txt 2024-04-24T23:16:16.001 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/get_dvb_firmware 2024-04-24T23:16:16.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/lmedm04.txt 2024-04-24T23:16:16.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/opera-firmware.txt 2024-04-24T23:16:16.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/readme.txt 2024-04-24T23:16:16.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/technisat.txt 2024-04-24T23:16:16.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/ttusb-dec.txt 2024-04-24T23:16:16.007 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dvb/udev.txt 2024-04-24T23:16:16.008 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/dynamic-debug-howto.txt 2024-04-24T23:16:16.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/early-userspace/ 2024-04-24T23:16:16.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/early-userspace/README 2024-04-24T23:16:16.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/early-userspace/buffer-format.txt 2024-04-24T23:16:16.011 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/edac.txt 2024-04-24T23:16:16.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/efi-stub.txt 2024-04-24T23:16:16.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/eisa.txt 2024-04-24T23:16:16.014 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/email-clients.txt 2024-04-24T23:16:16.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/extcon/ 2024-04-24T23:16:16.016 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/extcon/porting-android-switch-class 2024-04-24T23:16:16.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fault-injection/ 2024-04-24T23:16:16.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fault-injection/fault-injection.txt 2024-04-24T23:16:16.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fault-injection/notifier-error-inject.txt 2024-04-24T23:16:16.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fault-injection/provoke-crashes.txt 2024-04-24T23:16:16.020 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/ 2024-04-24T23:16:16.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/00-INDEX 2024-04-24T23:16:16.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/api.txt 2024-04-24T23:16:16.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/arkfb.txt 2024-04-24T23:16:16.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/aty128fb.txt 2024-04-24T23:16:16.024 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/cirrusfb.txt 2024-04-24T23:16:16.025 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/cmap_xfbdev.txt 2024-04-24T23:16:16.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/deferred_io.txt 2024-04-24T23:16:16.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/efifb.txt 2024-04-24T23:16:16.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/ep93xx-fb.txt 2024-04-24T23:16:16.030 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/fbcon.txt 2024-04-24T23:16:16.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/framebuffer.txt 2024-04-24T23:16:16.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/gxfb.txt 2024-04-24T23:16:16.033 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/intel810.txt 2024-04-24T23:16:16.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/intelfb.txt 2024-04-24T23:16:16.035 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/internals.txt 2024-04-24T23:16:16.036 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/lxfb.txt 2024-04-24T23:16:16.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/matroxfb.txt 2024-04-24T23:16:16.038 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/metronomefb.txt 2024-04-24T23:16:16.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/modedb.txt 2024-04-24T23:16:16.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/pvr2fb.txt 2024-04-24T23:16:16.042 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/pxafb.txt 2024-04-24T23:16:16.044 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/s3fb.txt 2024-04-24T23:16:16.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sa1100fb.txt 2024-04-24T23:16:16.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sh7760fb.txt 2024-04-24T23:16:16.048 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sisfb.txt 2024-04-24T23:16:16.049 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sm501.txt 2024-04-24T23:16:16.051 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/sstfb.txt 2024-04-24T23:16:16.052 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/tgafb.txt 2024-04-24T23:16:16.054 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/tridentfb.txt 2024-04-24T23:16:16.055 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/udlfb.txt 2024-04-24T23:16:16.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/uvesafb.txt 2024-04-24T23:16:16.058 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/vesafb.txt 2024-04-24T23:16:16.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/viafb.modes 2024-04-24T23:16:16.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/viafb.txt 2024-04-24T23:16:16.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fb/vt8623fb.txt 2024-04-24T23:16:16.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ 2024-04-24T23:16:16.064 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/.gitignore 2024-04-24T23:16:16.066 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/00-INDEX 2024-04-24T23:16:16.067 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/9p.txt 2024-04-24T23:16:16.069 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/Locking 2024-04-24T23:16:16.070 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/Makefile 2024-04-24T23:16:16.072 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/adfs.txt 2024-04-24T23:16:16.073 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/affs.txt 2024-04-24T23:16:16.074 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/afs.txt 2024-04-24T23:16:16.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/autofs4-mount-control.txt 2024-04-24T23:16:16.077 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/autofs4.txt 2024-04-24T23:16:16.078 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/automount-support.txt 2024-04-24T23:16:16.079 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/befs.txt 2024-04-24T23:16:16.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/bfs.txt 2024-04-24T23:16:16.081 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/btrfs.txt 2024-04-24T23:16:16.082 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/ 2024-04-24T23:16:16.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/backend-api.txt 2024-04-24T23:16:16.084 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/cachefiles.txt 2024-04-24T23:16:16.085 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/fscache.txt 2024-04-24T23:16:16.086 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/netfs-api.txt 2024-04-24T23:16:16.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/object.txt 2024-04-24T23:16:16.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/caching/operations.txt 2024-04-24T23:16:16.089 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ceph.txt 2024-04-24T23:16:16.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/ 2024-04-24T23:16:16.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/AUTHORS 2024-04-24T23:16:16.091 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/CHANGES 2024-04-24T23:16:16.092 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/README 2024-04-24T23:16:16.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/TODO 2024-04-24T23:16:16.094 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/cifs.txt 2024-04-24T23:16:16.095 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cifs/winucase_convert.pl 2024-04-24T23:16:16.096 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/coda.txt 2024-04-24T23:16:16.096 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/ 2024-04-24T23:16:16.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/Makefile 2024-04-24T23:16:16.098 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs.txt 2024-04-24T23:16:16.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_explicit.c 2024-04-24T23:16:16.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/configfs/configfs_example_macros.c 2024-04-24T23:16:16.100 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/cramfs.txt 2024-04-24T23:16:16.101 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/dax.txt 2024-04-24T23:16:16.102 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/debugfs.txt 2024-04-24T23:16:16.103 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/devpts.txt 2024-04-24T23:16:16.104 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/directory-locking 2024-04-24T23:16:16.104 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/dlmfs.txt 2024-04-24T23:16:16.105 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/dnotify.txt 2024-04-24T23:16:16.106 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/dnotify_test.c 2024-04-24T23:16:16.107 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ecryptfs.txt 2024-04-24T23:16:16.108 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/efivarfs.txt 2024-04-24T23:16:16.109 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/exofs.txt 2024-04-24T23:16:16.110 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ext2.txt 2024-04-24T23:16:16.110 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ext3.txt 2024-04-24T23:16:16.111 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ext4.txt 2024-04-24T23:16:16.112 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/f2fs.txt 2024-04-24T23:16:16.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/fiemap.txt 2024-04-24T23:16:16.114 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/files.txt 2024-04-24T23:16:16.115 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/fuse.txt 2024-04-24T23:16:16.116 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/gfs2-glocks.txt 2024-04-24T23:16:16.117 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/gfs2-uevents.txt 2024-04-24T23:16:16.118 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/gfs2.txt 2024-04-24T23:16:16.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/hfs.txt 2024-04-24T23:16:16.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/hfsplus.txt 2024-04-24T23:16:16.120 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/hpfs.txt 2024-04-24T23:16:16.121 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/inotify.txt 2024-04-24T23:16:16.122 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/isofs.txt 2024-04-24T23:16:16.123 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/jfs.txt 2024-04-24T23:16:16.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/locks.txt 2024-04-24T23:16:16.125 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/logfs.txt 2024-04-24T23:16:16.126 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/mandatory-locking.txt 2024-04-24T23:16:16.127 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ncpfs.txt 2024-04-24T23:16:16.128 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/ 2024-04-24T23:16:16.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/00-INDEX 2024-04-24T23:16:16.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/Exporting 2024-04-24T23:16:16.131 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/fault_injection.txt 2024-04-24T23:16:16.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/idmapper.txt 2024-04-24T23:16:16.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/knfsd-stats.txt 2024-04-24T23:16:16.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs-rdma.txt 2024-04-24T23:16:16.134 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs.txt 2024-04-24T23:16:16.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfs41-server.txt 2024-04-24T23:16:16.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsd-admin-interfaces.txt 2024-04-24T23:16:16.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/nfsroot.txt 2024-04-24T23:16:16.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs-block-server.txt 2024-04-24T23:16:16.139 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/pnfs.txt 2024-04-24T23:16:16.140 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-cache.txt 2024-04-24T23:16:16.141 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nfs/rpc-server-gss.txt 2024-04-24T23:16:16.142 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/nilfs2.txt 2024-04-24T23:16:16.143 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ntfs.txt 2024-04-24T23:16:16.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ocfs2.txt 2024-04-24T23:16:16.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/omfs.txt 2024-04-24T23:16:16.145 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/overlayfs.txt 2024-04-24T23:16:16.146 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/path-lookup.txt 2024-04-24T23:16:16.147 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/ 2024-04-24T23:16:16.148 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/design_notes.txt 2024-04-24T23:16:16.149 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/info.txt 2024-04-24T23:16:16.150 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/pohmelfs/network_protocol.txt 2024-04-24T23:16:16.151 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/porting 2024-04-24T23:16:16.152 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/proc.txt 2024-04-24T23:16:16.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/qnx6.txt 2024-04-24T23:16:16.154 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/quota.txt 2024-04-24T23:16:16.155 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ramfs-rootfs-initramfs.txt 2024-04-24T23:16:16.156 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/relay.txt 2024-04-24T23:16:16.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/romfs.txt 2024-04-24T23:16:16.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/seq_file.txt 2024-04-24T23:16:16.159 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sharedsubtree.txt 2024-04-24T23:16:16.160 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/spufs.txt 2024-04-24T23:16:16.161 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/squashfs.txt 2024-04-24T23:16:16.162 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sysfs-pci.txt 2024-04-24T23:16:16.163 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sysfs-tagging.txt 2024-04-24T23:16:16.164 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sysfs.txt 2024-04-24T23:16:16.165 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/sysv-fs.txt 2024-04-24T23:16:16.165 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/tmpfs.txt 2024-04-24T23:16:16.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ubifs.txt 2024-04-24T23:16:16.167 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/udf.txt 2024-04-24T23:16:16.168 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/ufs.txt 2024-04-24T23:16:16.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/vfat.txt 2024-04-24T23:16:16.170 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/vfs.txt 2024-04-24T23:16:16.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/xfs-delayed-logging-design.txt 2024-04-24T23:16:16.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/xfs-self-describing-metadata.txt 2024-04-24T23:16:16.172 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/filesystems/xfs.txt 2024-04-24T23:16:16.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/firmware_class/ 2024-04-24T23:16:16.174 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/firmware_class/README 2024-04-24T23:16:16.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/firmware_class/hotplug-script 2024-04-24T23:16:16.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/flexible-arrays.txt 2024-04-24T23:16:16.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/ 2024-04-24T23:16:16.178 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/00-INDEX 2024-04-24T23:16:16.179 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/API.txt 2024-04-24T23:16:16.180 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/FMC-and-SDB.txt 2024-04-24T23:16:16.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/carrier.txt 2024-04-24T23:16:16.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/fmc-chardev.txt 2024-04-24T23:16:16.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/fmc-fakedev.txt 2024-04-24T23:16:16.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/fmc-trivial.txt 2024-04-24T23:16:16.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/fmc-write-eeprom.txt 2024-04-24T23:16:16.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/identifiers.txt 2024-04-24T23:16:16.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/mezzanine.txt 2024-04-24T23:16:16.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/fmc/parameters.txt 2024-04-24T23:16:16.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/ 2024-04-24T23:16:16.189 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/README.txt 2024-04-24T23:16:16.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/atomic-ops.txt 2024-04-24T23:16:16.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/booting.txt 2024-04-24T23:16:16.191 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/clock.txt 2024-04-24T23:16:16.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/configuring.txt 2024-04-24T23:16:16.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/features.txt 2024-04-24T23:16:16.194 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/gdbinit 2024-04-24T23:16:16.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/gdbstub.txt 2024-04-24T23:16:16.196 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/kernel-ABI.txt 2024-04-24T23:16:16.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/frv/mmu-layout.txt 2024-04-24T23:16:16.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/futex-requeue-pi.txt 2024-04-24T23:16:16.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gcov.txt 2024-04-24T23:16:16.200 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gdb-kernel-debugging.txt 2024-04-24T23:16:16.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/ 2024-04-24T23:16:16.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/00-INDEX 2024-04-24T23:16:16.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/board.txt 2024-04-24T23:16:16.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/consumer.txt 2024-04-24T23:16:16.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/driver.txt 2024-04-24T23:16:16.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/gpio-legacy.txt 2024-04-24T23:16:16.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/gpio.txt 2024-04-24T23:16:16.207 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/gpio/sysfs.txt 2024-04-24T23:16:16.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/ 2024-04-24T23:16:16.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/hid-sensor.txt 2024-04-24T23:16:16.210 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/hid-transport.txt 2024-04-24T23:16:16.210 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/hiddev.txt 2024-04-24T23:16:16.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/hidraw.txt 2024-04-24T23:16:16.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hid/uhid.txt 2024-04-24T23:16:16.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/highuid.txt 2024-04-24T23:16:16.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hsi.txt 2024-04-24T23:16:16.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hw_random.txt 2024-04-24T23:16:16.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ 2024-04-24T23:16:16.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ab8500 2024-04-24T23:16:16.217 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/abituguru 2024-04-24T23:16:16.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/abituguru-datasheet 2024-04-24T23:16:16.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/abituguru3 2024-04-24T23:16:16.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/abx500 2024-04-24T23:16:16.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/acpi_power_meter 2024-04-24T23:16:16.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ad7314 2024-04-24T23:16:16.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adc128d818 2024-04-24T23:16:16.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1021 2024-04-24T23:16:16.225 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1025 2024-04-24T23:16:16.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1026 2024-04-24T23:16:16.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1031 2024-04-24T23:16:16.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm1275 2024-04-24T23:16:16.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adm9240 2024-04-24T23:16:16.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ads1015 2024-04-24T23:16:16.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ads7828 2024-04-24T23:16:16.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7410 2024-04-24T23:16:16.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7411 2024-04-24T23:16:16.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7462 2024-04-24T23:16:16.234 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7470 2024-04-24T23:16:16.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/adt7475 2024-04-24T23:16:16.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/amc6821 2024-04-24T23:16:16.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/asb100 2024-04-24T23:16:16.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/asc7621 2024-04-24T23:16:16.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/coretemp 2024-04-24T23:16:16.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/da9052 2024-04-24T23:16:16.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/da9055 2024-04-24T23:16:16.241 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/dme1737 2024-04-24T23:16:16.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ds1621 2024-04-24T23:16:16.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ds620 2024-04-24T23:16:16.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/emc1403 2024-04-24T23:16:16.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/emc2103 2024-04-24T23:16:16.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/emc6w201 2024-04-24T23:16:16.246 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/f71805f 2024-04-24T23:16:16.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/f71882fg 2024-04-24T23:16:16.248 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/fam15h_power 2024-04-24T23:16:16.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/g760a 2024-04-24T23:16:16.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/g762 2024-04-24T23:16:16.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/gl518sm 2024-04-24T23:16:16.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/hih6130 2024-04-24T23:16:16.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/htu21 2024-04-24T23:16:16.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/hwmon-kernel-api.txt 2024-04-24T23:16:16.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ibmaem 2024-04-24T23:16:16.256 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ibmpowernv 2024-04-24T23:16:16.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ina209 2024-04-24T23:16:16.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ina2xx 2024-04-24T23:16:16.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/it87 2024-04-24T23:16:16.260 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/jc42 2024-04-24T23:16:16.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/k10temp 2024-04-24T23:16:16.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/k8temp 2024-04-24T23:16:16.263 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lineage-pem 2024-04-24T23:16:16.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm25066 2024-04-24T23:16:16.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm63 2024-04-24T23:16:16.266 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm70 2024-04-24T23:16:16.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm73 2024-04-24T23:16:16.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm75 2024-04-24T23:16:16.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm77 2024-04-24T23:16:16.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm78 2024-04-24T23:16:16.270 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm80 2024-04-24T23:16:16.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm83 2024-04-24T23:16:16.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm85 2024-04-24T23:16:16.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm87 2024-04-24T23:16:16.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm90 2024-04-24T23:16:16.275 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm92 2024-04-24T23:16:16.276 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm93 2024-04-24T23:16:16.277 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm95234 2024-04-24T23:16:16.278 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/lm95245 2024-04-24T23:16:16.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:16 smithi112 ceph-mon[29186]: pgmap v408: 129 pgs: 129 active+clean; 21 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.7 MiB/s wr, 30 op/s 2024-04-24T23:16:16.279 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc2945 2024-04-24T23:16:16.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc2978 2024-04-24T23:16:16.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4151 2024-04-24T23:16:16.281 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4215 2024-04-24T23:16:16.282 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4245 2024-04-24T23:16:16.283 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4260 2024-04-24T23:16:16.284 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ltc4261 2024-04-24T23:16:16.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max16064 2024-04-24T23:16:16.286 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max16065 2024-04-24T23:16:16.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max1619 2024-04-24T23:16:16.288 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max1668 2024-04-24T23:16:16.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max197 2024-04-24T23:16:16.290 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max34440 2024-04-24T23:16:16.291 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max6639 2024-04-24T23:16:16.292 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max6642 2024-04-24T23:16:16.292 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max6650 2024-04-24T23:16:16.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max6697 2024-04-24T23:16:16.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/max8688 2024-04-24T23:16:16.295 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/mc13783-adc 2024-04-24T23:16:16.296 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/mcp3021 2024-04-24T23:16:16.297 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/menf21bmc 2024-04-24T23:16:16.298 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/nct6683 2024-04-24T23:16:16.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/nct6775 2024-04-24T23:16:16.300 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/nct7802 2024-04-24T23:16:16.301 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ntc_thermistor 2024-04-24T23:16:16.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pc87360 2024-04-24T23:16:16.303 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pc87427 2024-04-24T23:16:16.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pcf8591 2024-04-24T23:16:16.305 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pmbus 2024-04-24T23:16:16.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pmbus-core 2024-04-24T23:16:16.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/powr1220 2024-04-24T23:16:16.307 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/pwm-fan 2024-04-24T23:16:16.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sch5627 2024-04-24T23:16:16.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sch5636 2024-04-24T23:16:16.309 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sht15 2024-04-24T23:16:16.310 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sht21 2024-04-24T23:16:16.311 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/shtc1 2024-04-24T23:16:16.312 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sis5595 2024-04-24T23:16:16.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/smm665 2024-04-24T23:16:16.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/smsc47b397 2024-04-24T23:16:16.314 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/smsc47m1 2024-04-24T23:16:16.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/smsc47m192 2024-04-24T23:16:16.316 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/submitting-patches 2024-04-24T23:16:16.317 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/sysfs-interface 2024-04-24T23:16:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:16 smithi150 ceph-mon[32247]: pgmap v408: 129 pgs: 129 active+clean; 21 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.7 MiB/s wr, 30 op/s 2024-04-24T23:16:16.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/thmc50 2024-04-24T23:16:16.319 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tmp102 2024-04-24T23:16:16.320 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tmp103 2024-04-24T23:16:16.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tmp401 2024-04-24T23:16:16.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tmp421 2024-04-24T23:16:16.323 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/tps40422 2024-04-24T23:16:16.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/twl4030-madc-hwmon 2024-04-24T23:16:16.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ucd9000 2024-04-24T23:16:16.325 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/ucd9200 2024-04-24T23:16:16.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/userspace-tools 2024-04-24T23:16:16.327 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/vexpress 2024-04-24T23:16:16.328 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/via686a 2024-04-24T23:16:16.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/vt1211 2024-04-24T23:16:16.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83627ehf 2024-04-24T23:16:16.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83627hf 2024-04-24T23:16:16.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83781d 2024-04-24T23:16:16.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83791d 2024-04-24T23:16:16.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83792d 2024-04-24T23:16:16.334 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83793 2024-04-24T23:16:16.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83795 2024-04-24T23:16:16.336 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83l785ts 2024-04-24T23:16:16.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/w83l786ng 2024-04-24T23:16:16.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/wm831x 2024-04-24T23:16:16.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/wm8350 2024-04-24T23:16:16.339 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwmon/zl6100 2024-04-24T23:16:16.340 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/hwspinlock.txt 2024-04-24T23:16:16.341 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/ 2024-04-24T23:16:16.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/ 2024-04-24T23:16:16.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1535 2024-04-24T23:16:16.343 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali1563 2024-04-24T23:16:16.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ali15x3 2024-04-24T23:16:16.345 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd756 2024-04-24T23:16:16.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-amd8111 2024-04-24T23:16:16.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-diolan-u2c 2024-04-24T23:16:16.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-i801 2024-04-24T23:16:16.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ismt 2024-04-24T23:16:16.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-nforce2 2024-04-24T23:16:16.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-ocores 2024-04-24T23:16:16.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport 2024-04-24T23:16:16.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-parport-light 2024-04-24T23:16:16.352 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-pca-isa 2024-04-24T23:16:16.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-piix4 2024-04-24T23:16:16.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis5595 2024-04-24T23:16:16.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis630 2024-04-24T23:16:16.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-sis96x 2024-04-24T23:16:16.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-taos-evm 2024-04-24T23:16:16.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-via 2024-04-24T23:16:16.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/i2c-viapro 2024-04-24T23:16:16.358 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/busses/scx200_acb 2024-04-24T23:16:16.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/dev-interface 2024-04-24T23:16:16.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/fault-codes 2024-04-24T23:16:16.361 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/functionality 2024-04-24T23:16:16.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/i2c-protocol 2024-04-24T23:16:16.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/i2c-stub 2024-04-24T23:16:16.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/instantiating-devices 2024-04-24T23:16:16.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/muxes/ 2024-04-24T23:16:16.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/muxes/i2c-mux-gpio 2024-04-24T23:16:16.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/old-module-parameters 2024-04-24T23:16:16.366 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/smbus-protocol 2024-04-24T23:16:16.367 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/summary 2024-04-24T23:16:16.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/ten-bit-addresses 2024-04-24T23:16:16.369 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/upgrading-clients 2024-04-24T23:16:16.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2c/writing-clients 2024-04-24T23:16:16.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2o/ 2024-04-24T23:16:16.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2o/README 2024-04-24T23:16:16.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/i2o/ioctl 2024-04-24T23:16:16.373 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/ 2024-04-24T23:16:16.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/.gitignore 2024-04-24T23:16:16.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/IRQ-redir.txt 2024-04-24T23:16:16.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/Makefile 2024-04-24T23:16:16.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/README 2024-04-24T23:16:16.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/aliasing-test.c 2024-04-24T23:16:16.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/aliasing.txt 2024-04-24T23:16:16.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:16 smithi187 ceph-mon[33292]: pgmap v408: 129 pgs: 129 active+clean; 21 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 1.7 MiB/s wr, 30 op/s 2024-04-24T23:16:16.579 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/efirtc.txt 2024-04-24T23:16:16.580 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/err_inject.txt 2024-04-24T23:16:16.582 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/fsys.txt 2024-04-24T23:16:16.583 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/mca.txt 2024-04-24T23:16:16.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/serial.txt 2024-04-24T23:16:16.585 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ia64/xen.txt 2024-04-24T23:16:16.587 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ 2024-04-24T23:16:16.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/00-INDEX 2024-04-24T23:16:16.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-cd.1994-2004 2024-04-24T23:16:16.594 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-floppy.1996-2002 2024-04-24T23:16:16.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ChangeLog.ide-tape.1995-2002 2024-04-24T23:16:16.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ide-tape.txt 2024-04-24T23:16:16.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/ide.txt 2024-04-24T23:16:16.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ide/warm-plug-howto.txt 2024-04-24T23:16:16.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/ 2024-04-24T23:16:16.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/core_locking.txt 2024-04-24T23:16:16.601 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/ipoib.txt 2024-04-24T23:16:16.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/sysfs.txt 2024-04-24T23:16:16.603 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/user_mad.txt 2024-04-24T23:16:16.604 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/infiniband/user_verbs.txt 2024-04-24T23:16:16.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/init.txt 2024-04-24T23:16:16.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/initrd.txt 2024-04-24T23:16:16.607 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/ 2024-04-24T23:16:16.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/alps.txt 2024-04-24T23:16:16.609 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/amijoy.txt 2024-04-24T23:16:16.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/appletouch.txt 2024-04-24T23:16:16.611 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/atarikbd.txt 2024-04-24T23:16:16.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/bcm5974.txt 2024-04-24T23:16:16.613 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/cd32.txt 2024-04-24T23:16:16.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/cma3000_d0x.txt 2024-04-24T23:16:16.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/cs461x.txt 2024-04-24T23:16:16.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/edt-ft5x06.txt 2024-04-24T23:16:16.616 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/elantech.txt 2024-04-24T23:16:16.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/event-codes.txt 2024-04-24T23:16:16.619 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/ff.txt 2024-04-24T23:16:16.619 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/gamepad.txt 2024-04-24T23:16:16.620 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/gameport-programming.txt 2024-04-24T23:16:16.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/gpio-tilt.txt 2024-04-24T23:16:16.622 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/iforce-protocol.txt 2024-04-24T23:16:16.623 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/input-programming.txt 2024-04-24T23:16:16.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/input.txt 2024-04-24T23:16:16.625 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/interactive.fig 2024-04-24T23:16:16.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/joystick-api.txt 2024-04-24T23:16:16.627 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/joystick-parport.txt 2024-04-24T23:16:16.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/joystick.txt 2024-04-24T23:16:16.629 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/multi-touch-protocol.txt 2024-04-24T23:16:16.630 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/notifier.txt 2024-04-24T23:16:16.631 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/ntrig.txt 2024-04-24T23:16:16.632 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/rotary-encoder.txt 2024-04-24T23:16:16.633 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/sentelic.txt 2024-04-24T23:16:16.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/shape.fig 2024-04-24T23:16:16.635 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/walkera0701.txt 2024-04-24T23:16:16.636 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/xpad.txt 2024-04-24T23:16:16.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/input/yealink.txt 2024-04-24T23:16:16.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/intel_txt.txt 2024-04-24T23:16:16.639 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/io-mapping.txt 2024-04-24T23:16:16.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/io_ordering.txt 2024-04-24T23:16:16.641 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/ 2024-04-24T23:16:16.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/00-INDEX 2024-04-24T23:16:16.643 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/botching-up-ioctls.txt 2024-04-24T23:16:16.644 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/cdrom.txt 2024-04-24T23:16:16.645 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/hdio.txt 2024-04-24T23:16:16.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/ioctl-decoding.txt 2024-04-24T23:16:16.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ioctl/ioctl-number.txt 2024-04-24T23:16:16.648 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/iostats.txt 2024-04-24T23:16:16.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/irqflags-tracing.txt 2024-04-24T23:16:16.650 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isapnp.txt 2024-04-24T23:16:16.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/ 2024-04-24T23:16:16.652 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/00-INDEX 2024-04-24T23:16:16.653 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/CREDITS 2024-04-24T23:16:16.654 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/HiSax.cert 2024-04-24T23:16:16.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/INTERFACE 2024-04-24T23:16:16.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.CAPI 2024-04-24T23:16:16.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/INTERFACE.fax 2024-04-24T23:16:16.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README 2024-04-24T23:16:16.658 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.FAQ 2024-04-24T23:16:16.659 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.HiSax 2024-04-24T23:16:16.660 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.act2000 2024-04-24T23:16:16.661 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.audio 2024-04-24T23:16:16.662 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.avmb1 2024-04-24T23:16:16.662 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.concap 2024-04-24T23:16:16.663 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.diversion 2024-04-24T23:16:16.664 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.fax 2024-04-24T23:16:16.665 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.gigaset 2024-04-24T23:16:16.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.hfc-pci 2024-04-24T23:16:16.667 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.hysdn 2024-04-24T23:16:16.667 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.icn 2024-04-24T23:16:16.668 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.mISDN 2024-04-24T23:16:16.670 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.pcbit 2024-04-24T23:16:16.671 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.sc 2024-04-24T23:16:16.672 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.syncppp 2024-04-24T23:16:16.672 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/README.x25 2024-04-24T23:16:16.673 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/isdn/syncPPP.FAQ 2024-04-24T23:16:16.674 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/ 2024-04-24T23:16:16.675 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/HOWTO 2024-04-24T23:16:16.676 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/SubmitChecklist 2024-04-24T23:16:16.677 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/SubmittingPatches 2024-04-24T23:16:16.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/stable_api_nonsense.txt 2024-04-24T23:16:16.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ja_JP/stable_kernel_rules.txt 2024-04-24T23:16:16.680 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/java.txt 2024-04-24T23:16:16.681 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kasan.txt 2024-04-24T23:16:16.682 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/ 2024-04-24T23:16:16.683 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/00-INDEX 2024-04-24T23:16:16.684 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/headers_install.txt 2024-04-24T23:16:16.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/kbuild.txt 2024-04-24T23:16:16.686 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/kconfig-language.txt 2024-04-24T23:16:16.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/kconfig.txt 2024-04-24T23:16:16.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/makefiles.txt 2024-04-24T23:16:16.689 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kbuild/modules.txt 2024-04-24T23:16:16.690 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kdump/ 2024-04-24T23:16:16.691 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kdump/gdbmacros.txt 2024-04-24T23:16:16.692 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kdump/kdump.txt 2024-04-24T23:16:16.693 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kernel-doc-nano-HOWTO.txt 2024-04-24T23:16:16.694 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kernel-docs.txt 2024-04-24T23:16:16.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kernel-parameters.txt 2024-04-24T23:16:16.697 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kernel-per-CPU-kthreads.txt 2024-04-24T23:16:16.698 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kmemcheck.txt 2024-04-24T23:16:16.699 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kmemleak.txt 2024-04-24T23:16:16.700 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ko_KR/ 2024-04-24T23:16:16.701 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ko_KR/HOWTO 2024-04-24T23:16:16.702 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ko_KR/stable_api_nonsense.txt 2024-04-24T23:16:16.703 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kobject.txt 2024-04-24T23:16:16.704 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kprobes.txt 2024-04-24T23:16:16.705 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kref.txt 2024-04-24T23:16:16.706 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/kselftest.txt 2024-04-24T23:16:16.707 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/ 2024-04-24T23:16:16.708 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/.gitignore 2024-04-24T23:16:16.709 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/00-INDEX 2024-04-24T23:16:16.710 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/Makefile 2024-04-24T23:16:16.711 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/asus-laptop.txt 2024-04-24T23:16:16.712 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/disk-shock-protection.txt 2024-04-24T23:16:16.713 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/dslm.c 2024-04-24T23:16:16.714 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/freefall.c 2024-04-24T23:16:16.715 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/laptop-mode.txt 2024-04-24T23:16:16.716 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/sony-laptop.txt 2024-04-24T23:16:16.717 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/sonypi.txt 2024-04-24T23:16:16.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/thinkpad-acpi.txt 2024-04-24T23:16:16.718 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/laptops/toshiba_haps.txt 2024-04-24T23:16:16.719 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ldm.txt 2024-04-24T23:16:16.720 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/ 2024-04-24T23:16:16.721 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/00-INDEX 2024-04-24T23:16:16.722 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-blinkm.txt 2024-04-24T23:16:16.723 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-class.txt 2024-04-24T23:16:16.724 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lm3556.txt 2024-04-24T23:16:16.725 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp3944.txt 2024-04-24T23:16:16.726 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp5521.txt 2024-04-24T23:16:16.727 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp5523.txt 2024-04-24T23:16:16.728 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp5562.txt 2024-04-24T23:16:16.729 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/leds-lp55xx.txt 2024-04-24T23:16:16.730 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/ledtrig-oneshot.txt 2024-04-24T23:16:16.731 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/leds/ledtrig-transient.txt 2024-04-24T23:16:16.732 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/local_ops.txt 2024-04-24T23:16:16.733 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/ 2024-04-24T23:16:16.734 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/00-INDEX 2024-04-24T23:16:16.734 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/lglock.txt 2024-04-24T23:16:16.735 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/lockdep-design.txt 2024-04-24T23:16:16.736 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/lockstat.txt 2024-04-24T23:16:16.737 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/locktorture.txt 2024-04-24T23:16:16.738 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/mutex-design.txt 2024-04-24T23:16:16.739 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/rt-mutex-design.txt 2024-04-24T23:16:16.740 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/rt-mutex.txt 2024-04-24T23:16:16.741 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/spinlocks.txt 2024-04-24T23:16:16.742 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/locking/ww-mutex-design.txt 2024-04-24T23:16:16.742 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/lockup-watchdogs.txt 2024-04-24T23:16:16.744 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/logo.gif 2024-04-24T23:16:16.745 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/logo.txt 2024-04-24T23:16:16.746 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/lzo.txt 2024-04-24T23:16:16.747 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/m68k/ 2024-04-24T23:16:16.748 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/m68k/00-INDEX 2024-04-24T23:16:16.749 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/m68k/README.buddha 2024-04-24T23:16:16.750 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/m68k/kernel-options.txt 2024-04-24T23:16:16.751 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/magic-number.txt 2024-04-24T23:16:16.752 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mailbox.txt 2024-04-24T23:16:16.753 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/md.txt 2024-04-24T23:16:16.754 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/media-framework.txt 2024-04-24T23:16:16.755 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/memory-barriers.txt 2024-04-24T23:16:16.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/memory-devices/ 2024-04-24T23:16:16.836 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/memory-devices/ti-emif.txt 2024-04-24T23:16:16.837 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/memory-hotplug.txt 2024-04-24T23:16:16.838 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/metag/ 2024-04-24T23:16:16.839 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/metag/00-INDEX 2024-04-24T23:16:16.840 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/metag/kernel-ABI.txt 2024-04-24T23:16:16.841 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/ 2024-04-24T23:16:16.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/Makefile 2024-04-24T23:16:16.842 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mic_overview.txt 2024-04-24T23:16:16.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/ 2024-04-24T23:16:16.844 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/.gitignore 2024-04-24T23:16:16.845 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/Makefile 2024-04-24T23:16:16.846 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/micctrl 2024-04-24T23:16:16.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/mpss 2024-04-24T23:16:16.847 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.c 2024-04-24T23:16:16.848 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/mpssd.h 2024-04-24T23:16:16.849 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mic/mpssd/sysfs.c 2024-04-24T23:16:16.850 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mips/ 2024-04-24T23:16:16.851 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mips/00-INDEX 2024-04-24T23:16:16.852 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mips/AU1xxx_IDE.README 2024-04-24T23:16:16.853 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/ 2024-04-24T23:16:16.854 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/Makefile 2024-04-24T23:16:16.855 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/ad525x_dpot.txt 2024-04-24T23:16:16.856 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/apds990x.txt 2024-04-24T23:16:16.857 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/bh1770glc.txt 2024-04-24T23:16:16.858 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/c2port.txt 2024-04-24T23:16:16.859 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/eeprom 2024-04-24T23:16:16.860 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/ics932s401 2024-04-24T23:16:16.861 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/isl29003 2024-04-24T23:16:16.862 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/lis3lv02d 2024-04-24T23:16:16.863 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/max6875 2024-04-24T23:16:16.864 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/ 2024-04-24T23:16:16.865 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/.gitignore 2024-04-24T23:16:16.866 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/Makefile 2024-04-24T23:16:16.867 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/TODO 2024-04-24T23:16:16.868 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-amt-version.c 2024-04-24T23:16:16.869 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei-client-bus.txt 2024-04-24T23:16:16.870 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/mei/mei.txt 2024-04-24T23:16:16.871 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/misc-devices/spear-pcie-gadget.txt 2024-04-24T23:16:16.871 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/ 2024-04-24T23:16:16.872 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/00-INDEX 2024-04-24T23:16:16.874 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/mmc-async-req.txt 2024-04-24T23:16:16.876 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-attrs.txt 2024-04-24T23:16:16.877 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mmc/mmc-dev-parts.txt 2024-04-24T23:16:16.878 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mn10300/ 2024-04-24T23:16:16.880 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mn10300/ABI.txt 2024-04-24T23:16:16.881 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mn10300/compartmentalisation.txt 2024-04-24T23:16:16.882 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/module-signing.txt 2024-04-24T23:16:16.884 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mono.txt 2024-04-24T23:16:16.885 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/ 2024-04-24T23:16:16.887 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/nand/ 2024-04-24T23:16:16.888 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/nand/pxa3xx-nand.txt 2024-04-24T23:16:16.889 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/nand_ecc.txt 2024-04-24T23:16:16.891 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/mtd/spi-nor.txt 2024-04-24T23:16:16.892 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/namespaces/ 2024-04-24T23:16:16.893 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/namespaces/compatibility-list.txt 2024-04-24T23:16:16.895 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/namespaces/resource-control.txt 2024-04-24T23:16:16.896 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/ 2024-04-24T23:16:16.898 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/00-INDEX 2024-04-24T23:16:16.899 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/cipso_ipv4.txt 2024-04-24T23:16:16.901 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/draft-ietf-cipso-ipsecurity-01.txt 2024-04-24T23:16:16.902 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/introduction.txt 2024-04-24T23:16:16.903 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/netlabel/lsm_interface.txt 2024-04-24T23:16:16.905 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ 2024-04-24T23:16:16.906 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/00-INDEX 2024-04-24T23:16:16.907 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/3c509.txt 2024-04-24T23:16:16.909 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/6pack.txt 2024-04-24T23:16:16.910 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/LICENSE.qla3xxx 2024-04-24T23:16:16.912 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlcnic 2024-04-24T23:16:16.913 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/LICENSE.qlge 2024-04-24T23:16:16.914 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/Makefile 2024-04-24T23:16:16.916 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/PLIP.txt 2024-04-24T23:16:16.917 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/README.ipw2100 2024-04-24T23:16:16.919 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/README.ipw2200 2024-04-24T23:16:16.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/README.sb1000 2024-04-24T23:16:16.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/alias.txt 2024-04-24T23:16:16.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/altera_tse.txt 2024-04-24T23:16:16.923 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/arcnet-hardware.txt 2024-04-24T23:16:16.925 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/arcnet.txt 2024-04-24T23:16:16.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/atm.txt 2024-04-24T23:16:16.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ax25.txt 2024-04-24T23:16:16.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/batman-adv.txt 2024-04-24T23:16:16.928 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/baycom.txt 2024-04-24T23:16:16.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/bonding.txt 2024-04-24T23:16:16.931 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/bridge.txt 2024-04-24T23:16:16.932 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/caif/ 2024-04-24T23:16:16.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/caif/Linux-CAIF.txt 2024-04-24T23:16:16.933 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/caif/README 2024-04-24T23:16:16.934 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/caif/spi_porting.txt 2024-04-24T23:16:16.935 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/can.txt 2024-04-24T23:16:16.936 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cdc_mbim.txt 2024-04-24T23:16:16.937 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cops.txt 2024-04-24T23:16:16.938 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cs89x0.txt 2024-04-24T23:16:16.939 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cxacru-cf.py 2024-04-24T23:16:16.941 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cxacru.txt 2024-04-24T23:16:16.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/cxgb.txt 2024-04-24T23:16:16.942 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dccp.txt 2024-04-24T23:16:16.943 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dctcp.txt 2024-04-24T23:16:16.944 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/de4x5.txt 2024-04-24T23:16:16.945 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/decnet.txt 2024-04-24T23:16:16.946 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dl2k.txt 2024-04-24T23:16:16.947 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dm9000.txt 2024-04-24T23:16:16.948 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dmfe.txt 2024-04-24T23:16:16.949 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/dns_resolver.txt 2024-04-24T23:16:16.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/driver.txt 2024-04-24T23:16:16.950 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/e100.txt 2024-04-24T23:16:16.951 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/e1000.txt 2024-04-24T23:16:16.952 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/e1000e.txt 2024-04-24T23:16:16.953 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/eql.txt 2024-04-24T23:16:16.954 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/fib_trie.txt 2024-04-24T23:16:16.955 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/filter.txt 2024-04-24T23:16:16.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/fore200e.txt 2024-04-24T23:16:16.956 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/framerelay.txt 2024-04-24T23:16:16.957 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/gen_stats.txt 2024-04-24T23:16:16.958 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/generic-hdlc.txt 2024-04-24T23:16:16.959 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/generic_netlink.txt 2024-04-24T23:16:16.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/gianfar.txt 2024-04-24T23:16:16.960 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/i40e.txt 2024-04-24T23:16:16.961 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/i40evf.txt 2024-04-24T23:16:16.962 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ieee802154.txt 2024-04-24T23:16:16.963 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/igb.txt 2024-04-24T23:16:16.964 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/igbvf.txt 2024-04-24T23:16:16.965 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ip-sysctl.txt 2024-04-24T23:16:16.966 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ip_dynaddr.txt 2024-04-24T23:16:16.966 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipddp.txt 2024-04-24T23:16:16.967 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/iphase.txt 2024-04-24T23:16:16.968 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipsec.txt 2024-04-24T23:16:16.969 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipv6.txt 2024-04-24T23:16:16.970 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipvlan.txt 2024-04-24T23:16:16.971 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ipvs-sysctl.txt 2024-04-24T23:16:16.972 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/irda.txt 2024-04-24T23:16:16.973 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ixgb.txt 2024-04-24T23:16:16.974 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ixgbe.txt 2024-04-24T23:16:16.975 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ixgbevf.txt 2024-04-24T23:16:16.976 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/l2tp.txt 2024-04-24T23:16:16.977 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/lapb-module.txt 2024-04-24T23:16:16.978 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ltpc.txt 2024-04-24T23:16:16.979 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211-auth-assoc-deauth.txt 2024-04-24T23:16:16.980 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211-injection.txt 2024-04-24T23:16:16.981 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/ 2024-04-24T23:16:16.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/README 2024-04-24T23:16:16.982 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/hostapd.conf 2024-04-24T23:16:16.983 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/mac80211_hwsim/wpa_supplicant.conf 2024-04-24T23:16:16.984 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/multiqueue.txt 2024-04-24T23:16:16.985 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netconsole.txt 2024-04-24T23:16:16.986 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netdev-FAQ.txt 2024-04-24T23:16:16.987 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netdev-features.txt 2024-04-24T23:16:16.988 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netdevices.txt 2024-04-24T23:16:16.989 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netif-msg.txt 2024-04-24T23:16:16.990 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/netlink_mmap.txt 2024-04-24T23:16:16.991 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/nf_conntrack-sysctl.txt 2024-04-24T23:16:16.992 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/nfc.txt 2024-04-24T23:16:16.993 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/openvswitch.txt 2024-04-24T23:16:16.994 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/operstates.txt 2024-04-24T23:16:16.995 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/packet_mmap.txt 2024-04-24T23:16:16.996 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/phonet.txt 2024-04-24T23:16:16.997 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/phy.txt 2024-04-24T23:16:16.998 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/pktgen.txt 2024-04-24T23:16:16.999 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/policy-routing.txt 2024-04-24T23:16:17.000 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ppp_generic.txt 2024-04-24T23:16:17.001 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/proc_net_tcp.txt 2024-04-24T23:16:17.002 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/radiotap-headers.txt 2024-04-24T23:16:17.003 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/ray_cs.txt 2024-04-24T23:16:17.004 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/rds.txt 2024-04-24T23:16:17.005 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/regulatory.txt 2024-04-24T23:16:17.006 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/rxrpc.txt 2024-04-24T23:16:17.007 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/s2io.txt 2024-04-24T23:16:17.008 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/scaling.txt 2024-04-24T23:16:17.009 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/sctp.txt 2024-04-24T23:16:17.010 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/secid.txt 2024-04-24T23:16:17.011 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/skfp.txt 2024-04-24T23:16:17.012 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/smc9.txt 2024-04-24T23:16:17.013 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/spider_net.txt 2024-04-24T23:16:17.014 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/stmmac.txt 2024-04-24T23:16:17.015 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/switchdev.txt 2024-04-24T23:16:17.016 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tc-actions-env-rules.txt 2024-04-24T23:16:17.017 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tcp-thin.txt 2024-04-24T23:16:17.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tcp.txt 2024-04-24T23:16:17.018 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/team.txt 2024-04-24T23:16:17.019 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping.txt 2024-04-24T23:16:17.020 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/ 2024-04-24T23:16:17.021 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/.gitignore 2024-04-24T23:16:17.022 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/Makefile 2024-04-24T23:16:17.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/hwtstamp_config.c 2024-04-24T23:16:17.023 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/timestamping.c 2024-04-24T23:16:17.025 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/timestamping/txtimestamp.c 2024-04-24T23:16:17.026 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tlan.txt 2024-04-24T23:16:17.027 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tproxy.txt 2024-04-24T23:16:17.028 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/tuntap.txt 2024-04-24T23:16:17.029 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/udplite.txt 2024-04-24T23:16:17.030 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/vortex.txt 2024-04-24T23:16:17.031 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/vxge.txt 2024-04-24T23:16:17.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/vxlan.txt 2024-04-24T23:16:17.032 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/x25-iface.txt 2024-04-24T23:16:17.033 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/x25.txt 2024-04-24T23:16:17.034 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/xfrm_proc.txt 2024-04-24T23:16:17.035 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/xfrm_sync.txt 2024-04-24T23:16:17.036 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/xfrm_sysctl.txt 2024-04-24T23:16:17.037 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/networking/z8530drv.txt 2024-04-24T23:16:17.038 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nfc/ 2024-04-24T23:16:17.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nfc/nfc-hci.txt 2024-04-24T23:16:17.039 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nfc/nfc-pn544.txt 2024-04-24T23:16:17.040 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nios2/ 2024-04-24T23:16:17.041 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nios2/README 2024-04-24T23:16:17.042 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/nommu-mmap.txt 2024-04-24T23:16:17.045 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/numastat.txt 2024-04-24T23:16:17.046 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/oops-tracing.txt 2024-04-24T23:16:17.047 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/padata.txt 2024-04-24T23:16:17.047 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parisc/ 2024-04-24T23:16:17.047 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parisc/00-INDEX 2024-04-24T23:16:17.048 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parisc/debugging 2024-04-24T23:16:17.049 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parisc/registers 2024-04-24T23:16:17.050 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parport-lowlevel.txt 2024-04-24T23:16:17.051 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/parport.txt 2024-04-24T23:16:17.052 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/ 2024-04-24T23:16:17.053 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/.gitignore 2024-04-24T23:16:17.054 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/Makefile 2024-04-24T23:16:17.055 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/crc32hash.c 2024-04-24T23:16:17.056 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/devicetable.txt 2024-04-24T23:16:17.057 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/driver-changes.txt 2024-04-24T23:16:17.058 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/driver.txt 2024-04-24T23:16:17.059 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pcmcia/locking.txt 2024-04-24T23:16:17.060 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/percpu-rw-semaphore.txt 2024-04-24T23:16:17.061 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/phy.txt 2024-04-24T23:16:17.062 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/phy/ 2024-04-24T23:16:17.063 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/phy/samsung-usb2.txt 2024-04-24T23:16:17.064 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pi-futex.txt 2024-04-24T23:16:17.065 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pinctrl.txt 2024-04-24T23:16:17.066 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/platform/ 2024-04-24T23:16:17.067 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/platform/x86-laptop-drivers.txt 2024-04-24T23:16:17.068 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pnp.txt 2024-04-24T23:16:17.070 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/ 2024-04-24T23:16:17.071 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/00-INDEX 2024-04-24T23:16:17.072 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/apm-acpi.txt 2024-04-24T23:16:17.074 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/basic-pm-debugging.txt 2024-04-24T23:16:17.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/charger-manager.txt 2024-04-24T23:16:17.076 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/devices.txt 2024-04-24T23:16:17.078 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/drivers-testing.txt 2024-04-24T23:16:17.079 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/freezing-of-tasks.txt 2024-04-24T23:16:17.080 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/interface.txt 2024-04-24T23:16:17.082 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/notifiers.txt 2024-04-24T23:16:17.083 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/opp.txt 2024-04-24T23:16:17.084 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/pci.txt 2024-04-24T23:16:17.086 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/pm_qos_interface.txt 2024-04-24T23:16:17.087 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/power_supply_class.txt 2024-04-24T23:16:17.089 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/powercap/ 2024-04-24T23:16:17.090 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/powercap/powercap.txt 2024-04-24T23:16:17.091 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/ 2024-04-24T23:16:17.093 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/consumer.txt 2024-04-24T23:16:17.094 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/design.txt 2024-04-24T23:16:17.095 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/machine.txt 2024-04-24T23:16:17.097 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/overview.txt 2024-04-24T23:16:17.098 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/regulator/regulator.txt 2024-04-24T23:16:17.099 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/runtime_pm.txt 2024-04-24T23:16:17.100 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/s2ram.txt 2024-04-24T23:16:17.102 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/states.txt 2024-04-24T23:16:17.103 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/suspend-and-cpuhotplug.txt 2024-04-24T23:16:17.105 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/suspend-and-interrupts.txt 2024-04-24T23:16:17.106 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/swsusp-and-swap-files.txt 2024-04-24T23:16:17.108 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/swsusp-dmcrypt.txt 2024-04-24T23:16:17.109 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/swsusp.txt 2024-04-24T23:16:17.110 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/tricks.txt 2024-04-24T23:16:17.112 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/userland-swsusp.txt 2024-04-24T23:16:17.113 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/power/video.txt 2024-04-24T23:16:17.114 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/ 2024-04-24T23:16:17.116 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/00-INDEX 2024-04-24T23:16:17.117 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/bootwrapper.txt 2024-04-24T23:16:17.119 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/cpu_families.txt 2024-04-24T23:16:17.120 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/cpu_features.txt 2024-04-24T23:16:17.121 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/cxl.txt 2024-04-24T23:16:17.123 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/eeh-pci-error-recovery.txt 2024-04-24T23:16:17.124 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/firmware-assisted-dump.txt 2024-04-24T23:16:17.125 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/hvcs.txt 2024-04-24T23:16:17.127 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/mpc52xx.txt 2024-04-24T23:16:17.128 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/pmu-ebb.txt 2024-04-24T23:16:17.129 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/ptrace.txt 2024-04-24T23:16:17.130 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/qe_firmware.txt 2024-04-24T23:16:17.131 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/powerpc/transactional_memory.txt 2024-04-24T23:16:17.132 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pps/ 2024-04-24T23:16:17.133 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pps/pps.txt 2024-04-24T23:16:17.134 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/ 2024-04-24T23:16:17.135 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/.gitignore 2024-04-24T23:16:17.136 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/Makefile 2024-04-24T23:16:17.137 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-ctxt-sw-stress-test.c 2024-04-24T23:16:17.138 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-on-off-stress-test.c 2024-04-24T23:16:17.140 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/disable-tsc-test.c 2024-04-24T23:16:17.141 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/no_new_privs.txt 2024-04-24T23:16:17.142 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/prctl/seccomp_filter.txt 2024-04-24T23:16:17.143 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/preempt-locking.txt 2024-04-24T23:16:17.144 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/printk-formats.txt 2024-04-24T23:16:17.145 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pti/ 2024-04-24T23:16:17.147 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pti/pti_intel_mid.txt 2024-04-24T23:16:17.148 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/ 2024-04-24T23:16:17.149 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/.gitignore 2024-04-24T23:16:17.150 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/Makefile 2024-04-24T23:16:17.151 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/ptp.txt 2024-04-24T23:16:17.152 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/testptp.c 2024-04-24T23:16:17.153 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ptp/testptp.mk 2024-04-24T23:16:17.154 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/pwm.txt 2024-04-24T23:16:17.156 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/ramoops.txt 2024-04-24T23:16:17.157 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rapidio/ 2024-04-24T23:16:17.158 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rapidio/rapidio.txt 2024-04-24T23:16:17.159 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rapidio/sysfs.txt 2024-04-24T23:16:17.160 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rapidio/tsi721.txt 2024-04-24T23:16:17.161 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rbtree.txt 2024-04-24T23:16:17.163 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/remoteproc.txt 2024-04-24T23:16:17.164 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rfkill.txt 2024-04-24T23:16:17.165 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/robust-futex-ABI.txt 2024-04-24T23:16:17.166 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/robust-futexes.txt 2024-04-24T23:16:17.168 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rpmsg.txt 2024-04-24T23:16:17.169 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/rtc.txt 2024-04-24T23:16:17.170 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/ 2024-04-24T23:16:17.171 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/00-INDEX 2024-04-24T23:16:17.173 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/3270.ChangeLog 2024-04-24T23:16:17.174 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/3270.txt 2024-04-24T23:16:17.175 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/CommonIO 2024-04-24T23:16:17.176 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/DASD 2024-04-24T23:16:17.177 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/Debugging390.txt 2024-04-24T23:16:17.178 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/cds.txt 2024-04-24T23:16:17.180 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/config3270.sh 2024-04-24T23:16:17.181 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/driver-model.txt 2024-04-24T23:16:17.182 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/kvm.txt 2024-04-24T23:16:17.183 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/monreader.txt 2024-04-24T23:16:17.184 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/qeth.txt 2024-04-24T23:16:17.185 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/s390dbf.txt 2024-04-24T23:16:17.186 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/s390/zfcpdump.txt 2024-04-24T23:16:17.187 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/ 2024-04-24T23:16:17.188 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/00-INDEX 2024-04-24T23:16:17.190 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/completion.txt 2024-04-24T23:16:17.191 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-arch.txt 2024-04-24T23:16:17.192 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-bwc.txt 2024-04-24T23:16:17.193 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-deadline.txt 2024-04-24T23:16:17.194 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-design-CFS.txt 2024-04-24T23:16:17.195 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-domains.txt 2024-04-24T23:16:17.197 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-nice-design.txt 2024-04-24T23:16:17.198 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-rt-group.txt 2024-04-24T23:16:17.199 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scheduler/sched-stats.txt 2024-04-24T23:16:17.200 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ 2024-04-24T23:16:17.201 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/00-INDEX 2024-04-24T23:16:17.202 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/53c700.txt 2024-04-24T23:16:17.203 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/BusLogic.txt 2024-04-24T23:16:17.204 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.1992-1997 2024-04-24T23:16:17.205 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.arcmsr 2024-04-24T23:16:17.206 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ips 2024-04-24T23:16:17.208 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.lpfc 2024-04-24T23:16:17.209 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid 2024-04-24T23:16:17.211 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.megaraid_sas 2024-04-24T23:16:17.212 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.ncr53c8xx 2024-04-24T23:16:17.213 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx 2024-04-24T23:16:17.214 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ChangeLog.sym53c8xx_2 2024-04-24T23:16:17.215 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/FlashPoint.txt 2024-04-24T23:16:17.216 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/LICENSE.FlashPoint 2024-04-24T23:16:17.218 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla2xxx 2024-04-24T23:16:17.219 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/LICENSE.qla4xxx 2024-04-24T23:16:17.220 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/Mylex.txt 2024-04-24T23:16:17.221 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/NinjaSCSI.txt 2024-04-24T23:16:17.222 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/aacraid.txt 2024-04-24T23:16:17.223 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/advansys.txt 2024-04-24T23:16:17.224 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/aha152x.txt 2024-04-24T23:16:17.225 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/aic79xx.txt 2024-04-24T23:16:17.226 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/aic7xxx.txt 2024-04-24T23:16:17.228 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/arcmsr_spec.txt 2024-04-24T23:16:17.229 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/bfa.txt 2024-04-24T23:16:17.230 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/bnx2fc.txt 2024-04-24T23:16:17.231 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/cxgb3i.txt 2024-04-24T23:16:17.232 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/dc395x.txt 2024-04-24T23:16:17.233 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/dpti.txt 2024-04-24T23:16:17.235 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/dtc3x80.txt 2024-04-24T23:16:17.236 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/g_NCR5380.txt 2024-04-24T23:16:17.237 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/hpsa.txt 2024-04-24T23:16:17.238 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/hptiop.txt 2024-04-24T23:16:17.239 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/in2000.txt 2024-04-24T23:16:17.240 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/libsas.txt 2024-04-24T23:16:17.242 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/link_power_management_policy.txt 2024-04-24T23:16:17.243 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/lpfc.txt 2024-04-24T23:16:17.244 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/megaraid.txt 2024-04-24T23:16:17.245 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ncr53c8xx.txt 2024-04-24T23:16:17.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/osd.txt 2024-04-24T23:16:17.247 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/osst.txt 2024-04-24T23:16:17.249 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ppa.txt 2024-04-24T23:16:17.250 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/qlogicfas.txt 2024-04-24T23:16:17.251 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi-changer.txt 2024-04-24T23:16:17.252 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi-generic.txt 2024-04-24T23:16:17.253 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi-parameters.txt 2024-04-24T23:16:17.254 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi.txt 2024-04-24T23:16:17.255 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_eh.txt 2024-04-24T23:16:17.257 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_fc_transport.txt 2024-04-24T23:16:17.258 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_mid_low_api.txt 2024-04-24T23:16:17.259 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/ 2024-04-24T23:16:17.260 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/Makefile 2024-04-24T23:16:17.261 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/scsi_transport_srp/rport_state_diagram.dot 2024-04-24T23:16:17.262 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/st.txt 2024-04-24T23:16:17.264 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/sym53c500_cs.txt 2024-04-24T23:16:17.265 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/sym53c8xx_2.txt 2024-04-24T23:16:17.266 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/tmscsim.txt 2024-04-24T23:16:17.267 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/ufs.txt 2024-04-24T23:16:17.268 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/scsi/wd719x.txt 2024-04-24T23:16:17.269 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/ 2024-04-24T23:16:17.271 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/00-INDEX 2024-04-24T23:16:17.272 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/IMA-templates.txt 2024-04-24T23:16:17.273 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/LSM.txt 2024-04-24T23:16:17.274 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/SELinux.txt 2024-04-24T23:16:17.275 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/Smack.txt 2024-04-24T23:16:17.276 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/Yama.txt 2024-04-24T23:16:17.278 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/apparmor.txt 2024-04-24T23:16:17.279 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/credentials.txt 2024-04-24T23:16:17.280 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/keys-ecryptfs.txt 2024-04-24T23:16:17.281 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/keys-request-key.txt 2024-04-24T23:16:17.283 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/keys-trusted-encrypted.txt 2024-04-24T23:16:17.284 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/keys.txt 2024-04-24T23:16:17.285 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/security/tomoyo.txt 2024-04-24T23:16:17.286 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial-console.txt 2024-04-24T23:16:17.287 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/ 2024-04-24T23:16:17.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/00-INDEX 2024-04-24T23:16:17.289 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/README.cycladesZ 2024-04-24T23:16:17.291 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/driver 2024-04-24T23:16:17.292 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/moxa-smartio 2024-04-24T23:16:17.293 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/n_gsm.txt 2024-04-24T23:16:17.294 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/rocket.txt 2024-04-24T23:16:17.296 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/serial-rs485.txt 2024-04-24T23:16:17.297 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/serial/tty.txt 2024-04-24T23:16:17.298 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sgi-ioc4.txt 2024-04-24T23:16:17.299 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sh/ 2024-04-24T23:16:17.300 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sh/new-machine.txt 2024-04-24T23:16:17.302 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sh/register-banks.txt 2024-04-24T23:16:17.303 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/smsc_ece1099.txt 2024-04-24T23:16:17.304 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/ 2024-04-24T23:16:17.305 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/ 2024-04-24T23:16:17.306 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/ALSA-Configuration.txt 2024-04-24T23:16:17.307 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Audigy-mixer.txt 2024-04-24T23:16:17.308 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Audiophile-Usb.txt 2024-04-24T23:16:17.310 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Bt87x.txt 2024-04-24T23:16:17.311 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/CMIPCI.txt 2024-04-24T23:16:17.312 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Channel-Mapping-API.txt 2024-04-24T23:16:17.313 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/ControlNames.txt 2024-04-24T23:16:17.314 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Controls.txt 2024-04-24T23:16:17.315 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio-Models.txt 2024-04-24T23:16:17.316 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/HD-Audio.txt 2024-04-24T23:16:17.317 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Joystick.txt 2024-04-24T23:16:17.318 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/MIXART.txt 2024-04-24T23:16:17.319 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/OSS-Emulation.txt 2024-04-24T23:16:17.321 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/Procfile.txt 2024-04-24T23:16:17.322 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/README.maya44 2024-04-24T23:16:17.323 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/SB-Live-mixer.txt 2024-04-24T23:16:17.324 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/VIA82xx-mixer.txt 2024-04-24T23:16:17.325 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/alsa-parameters.txt 2024-04-24T23:16:17.326 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/compress_offload.txt 2024-04-24T23:16:17.327 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/emu10k1-jack.txt 2024-04-24T23:16:17.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/hda_codec.txt 2024-04-24T23:16:17.329 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/hdspm.txt 2024-04-24T23:16:17.330 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/powersave.txt 2024-04-24T23:16:17.331 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/seq_oss.html 2024-04-24T23:16:17.332 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/serial-u16550.txt 2024-04-24T23:16:17.333 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/ 2024-04-24T23:16:17.334 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DAI.txt 2024-04-24T23:16:17.335 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/DPCM.txt 2024-04-24T23:16:17.336 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/clocking.txt 2024-04-24T23:16:17.337 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/codec.txt 2024-04-24T23:16:17.338 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/dapm.txt 2024-04-24T23:16:17.339 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/jack.txt 2024-04-24T23:16:17.340 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/machine.txt 2024-04-24T23:16:17.341 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/overview.txt 2024-04-24T23:16:17.342 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/platform.txt 2024-04-24T23:16:17.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/alsa/soc/pops_clicks.txt 2024-04-24T23:16:17.344 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ 2024-04-24T23:16:17.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ALS 2024-04-24T23:16:17.346 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/AudioExcelDSP16 2024-04-24T23:16:17.347 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/CMI8330 2024-04-24T23:16:17.348 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ESS 2024-04-24T23:16:17.349 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ESS1868 2024-04-24T23:16:17.350 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/Introduction 2024-04-24T23:16:17.351 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/MultiSound 2024-04-24T23:16:17.352 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/OPL3 2024-04-24T23:16:17.353 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/Opti 2024-04-24T23:16:17.354 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/PAS16 2024-04-24T23:16:17.355 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/PSS 2024-04-24T23:16:17.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/PSS-updates 2024-04-24T23:16:17.356 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/README.OSS 2024-04-24T23:16:17.357 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/README.modules 2024-04-24T23:16:17.358 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/README.ymfsb 2024-04-24T23:16:17.359 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/SoundPro 2024-04-24T23:16:17.360 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/Soundblaster 2024-04-24T23:16:17.361 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/Tropez+ 2024-04-24T23:16:17.362 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/VIBRA16 2024-04-24T23:16:17.363 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/WaveArtist 2024-04-24T23:16:17.364 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/btaudio 2024-04-24T23:16:17.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/mwave 2024-04-24T23:16:17.365 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/oss-parameters.txt 2024-04-24T23:16:17.366 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sound/oss/ultrasound 2024-04-24T23:16:17.368 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sparse.txt 2024-04-24T23:16:17.369 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/ 2024-04-24T23:16:17.370 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/.gitignore 2024-04-24T23:16:17.371 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/00-INDEX 2024-04-24T23:16:17.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/Makefile 2024-04-24T23:16:17.372 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/butterfly 2024-04-24T23:16:17.374 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/ep93xx_spi 2024-04-24T23:16:17.375 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/pxa2xx 2024-04-24T23:16:17.376 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spi-lm70llp 2024-04-24T23:16:17.377 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spi-sc18is602 2024-04-24T23:16:17.378 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spi-summary 2024-04-24T23:16:17.379 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spidev 2024-04-24T23:16:17.380 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spidev_fdx.c 2024-04-24T23:16:17.381 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/spi/spidev_test.c 2024-04-24T23:16:17.382 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/stable_api_nonsense.txt 2024-04-24T23:16:17.383 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/stable_kernel_rules.txt 2024-04-24T23:16:17.384 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/static-keys.txt 2024-04-24T23:16:17.385 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/svga.txt 2024-04-24T23:16:17.386 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/ 2024-04-24T23:16:17.387 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/00-INDEX 2024-04-24T23:16:17.388 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/README 2024-04-24T23:16:17.389 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/abi.txt 2024-04-24T23:16:17.390 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/fs.txt 2024-04-24T23:16:17.391 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/kernel.txt 2024-04-24T23:16:17.392 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/net.txt 2024-04-24T23:16:17.393 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/sunrpc.txt 2024-04-24T23:16:17.394 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysctl/vm.txt 2024-04-24T23:16:17.395 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysfs-rules.txt 2024-04-24T23:16:17.396 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/sysrq.txt 2024-04-24T23:16:17.397 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/target/ 2024-04-24T23:16:17.398 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.py 2024-04-24T23:16:17.399 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/target/tcm_mod_builder.txt 2024-04-24T23:16:17.400 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/target/tcmu-design.txt 2024-04-24T23:16:17.401 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/ 2024-04-24T23:16:17.402 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/cpu-cooling-api.txt 2024-04-24T23:16:17.403 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal 2024-04-24T23:16:17.404 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/exynos_thermal_emulation 2024-04-24T23:16:17.405 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/intel_powerclamp.txt 2024-04-24T23:16:17.406 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/nouveau_thermal 2024-04-24T23:16:17.407 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/sysfs-api.txt 2024-04-24T23:16:17.408 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/thermal/x86_pkg_temperature_thermal 2024-04-24T23:16:17.409 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/this_cpu_ops.txt 2024-04-24T23:16:17.410 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/ 2024-04-24T23:16:17.411 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/.gitignore 2024-04-24T23:16:17.412 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/00-INDEX 2024-04-24T23:16:17.413 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/Makefile 2024-04-24T23:16:17.414 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/NO_HZ.txt 2024-04-24T23:16:17.415 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/highres.txt 2024-04-24T23:16:17.416 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/hpet.txt 2024-04-24T23:16:17.417 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/hpet_example.c 2024-04-24T23:16:17.418 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/hrtimers.txt 2024-04-24T23:16:17.419 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/timekeeping.txt 2024-04-24T23:16:17.420 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/timer_stats.txt 2024-04-24T23:16:17.421 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/timers/timers-howto.txt 2024-04-24T23:16:17.422 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/tpm/ 2024-04-24T23:16:17.423 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/tpm/xen-tpmfront.txt 2024-04-24T23:16:17.424 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/ 2024-04-24T23:16:17.425 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/coresight.txt 2024-04-24T23:16:17.425 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/events-kmem.txt 2024-04-24T23:16:17.427 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/events-nmi.txt 2024-04-24T23:16:17.428 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/events-power.txt 2024-04-24T23:16:17.429 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/events.txt 2024-04-24T23:16:17.430 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/ftrace-design.txt 2024-04-24T23:16:17.431 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/ftrace.txt 2024-04-24T23:16:17.432 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/function-graph-fold.vim 2024-04-24T23:16:17.433 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/kprobetrace.txt 2024-04-24T23:16:17.434 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/mmiotrace.txt 2024-04-24T23:16:17.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/postprocess/ 2024-04-24T23:16:17.435 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-pagealloc-postprocess.pl 2024-04-24T23:16:17.436 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/postprocess/trace-vmscan-postprocess.pl 2024-04-24T23:16:17.437 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/ring-buffer-design.txt 2024-04-24T23:16:17.438 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/tracepoint-analysis.txt 2024-04-24T23:16:17.439 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/tracepoints.txt 2024-04-24T23:16:17.440 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/trace/uprobetracer.txt 2024-04-24T23:16:17.441 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/unaligned-memory-access.txt 2024-04-24T23:16:17.442 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/unicode.txt 2024-04-24T23:16:17.443 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/unshare.txt 2024-04-24T23:16:17.444 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/ 2024-04-24T23:16:17.445 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/CREDITS 2024-04-24T23:16:17.446 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/URB.txt 2024-04-24T23:16:17.447 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/WUSB-Design-overview.txt 2024-04-24T23:16:17.448 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/acm.txt 2024-04-24T23:16:17.449 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/anchors.txt 2024-04-24T23:16:17.450 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/authorization.txt 2024-04-24T23:16:17.453 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/bulk-streams.txt 2024-04-24T23:16:17.454 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/callbacks.txt 2024-04-24T23:16:17.454 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/chipidea.txt 2024-04-24T23:16:17.455 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/dma.txt 2024-04-24T23:16:17.455 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/dwc3.txt 2024-04-24T23:16:17.455 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/ehci.txt 2024-04-24T23:16:17.456 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/error-codes.txt 2024-04-24T23:16:17.457 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/functionfs.txt 2024-04-24T23:16:17.458 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget-testing.txt 2024-04-24T23:16:17.459 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_configfs.txt 2024-04-24T23:16:17.460 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_hid.txt 2024-04-24T23:16:17.461 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_multi.txt 2024-04-24T23:16:17.462 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_printer.txt 2024-04-24T23:16:17.463 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/gadget_serial.txt 2024-04-24T23:16:17.464 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/hotplug.txt 2024-04-24T23:16:17.465 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/iuu_phoenix.txt 2024-04-24T23:16:17.466 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/linux-cdc-acm.inf 2024-04-24T23:16:17.467 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/linux.inf 2024-04-24T23:16:17.468 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/mass-storage.txt 2024-04-24T23:16:17.469 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/misc_usbsevseg.txt 2024-04-24T23:16:17.470 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/mtouchusb.txt 2024-04-24T23:16:17.471 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/ohci.txt 2024-04-24T23:16:17.472 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/persist.txt 2024-04-24T23:16:17.473 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/power-management.txt 2024-04-24T23:16:17.474 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/proc_usb_info.txt 2024-04-24T23:16:17.475 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/rio.txt 2024-04-24T23:16:17.476 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/usb-help.txt 2024-04-24T23:16:17.477 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/usb-serial.txt 2024-04-24T23:16:17.478 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/usbmon.txt 2024-04-24T23:16:17.479 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/usb/wusb-cbaf 2024-04-24T23:16:17.480 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/ 2024-04-24T23:16:17.481 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/.gitignore 2024-04-24T23:16:17.482 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/Makefile 2024-04-24T23:16:17.483 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/parse_vdso.c 2024-04-24T23:16:17.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/vdso_standalone_test_x86.c 2024-04-24T23:16:17.484 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vDSO/vdso_test.c 2024-04-24T23:16:17.486 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vfio.txt 2024-04-24T23:16:17.487 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vgaarbiter.txt 2024-04-24T23:16:17.488 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video-output.txt 2024-04-24T23:16:17.489 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/ 2024-04-24T23:16:17.490 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/4CCs.txt 2024-04-24T23:16:17.491 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/API.html 2024-04-24T23:16:17.492 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.au0828 2024-04-24T23:16:17.493 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.bttv 2024-04-24T23:16:17.494 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx23885 2024-04-24T23:16:17.495 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.cx88 2024-04-24T23:16:17.496 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.em28xx 2024-04-24T23:16:17.497 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.ivtv 2024-04-24T23:16:17.498 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7134 2024-04-24T23:16:17.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.saa7164 2024-04-24T23:16:17.499 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tm6000 2024-04-24T23:16:17.500 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.tuner 2024-04-24T23:16:17.501 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/CARDLIST.usbvision 2024-04-24T23:16:17.502 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/Makefile 2024-04-24T23:16:17.503 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.cpia2 2024-04-24T23:16:17.504 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.cx88 2024-04-24T23:16:17.505 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.davinci-vpbe 2024-04-24T23:16:17.506 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.ir 2024-04-24T23:16:17.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.ivtv 2024-04-24T23:16:17.507 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.pvrusb2 2024-04-24T23:16:17.508 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/README.saa7134 2024-04-24T23:16:17.509 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/Zoran 2024-04-24T23:16:17.510 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/ 2024-04-24T23:16:17.511 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/CONTRIBUTORS 2024-04-24T23:16:17.512 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Cards 2024-04-24T23:16:17.513 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/ICs 2024-04-24T23:16:17.514 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Insmod-options 2024-04-24T23:16:17.515 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/MAKEDEV 2024-04-24T23:16:17.516 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modprobe.conf 2024-04-24T23:16:17.517 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Modules.conf 2024-04-24T23:16:17.518 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/PROBLEMS 2024-04-24T23:16:17.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/README 2024-04-24T23:16:17.519 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.WINVIEW 2024-04-24T23:16:17.520 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.freeze 2024-04-24T23:16:17.521 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/README.quirks 2024-04-24T23:16:17.522 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Sound-FAQ 2024-04-24T23:16:17.523 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Specs 2024-04-24T23:16:17.524 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/THANKS 2024-04-24T23:16:17.525 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/bttv/Tuners 2024-04-24T23:16:17.526 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cafe_ccic 2024-04-24T23:16:17.527 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cpia2_overview.txt 2024-04-24T23:16:17.528 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx18.txt 2024-04-24T23:16:17.529 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/ 2024-04-24T23:16:17.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.hm12 2024-04-24T23:16:17.530 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/README.vbi 2024-04-24T23:16:17.531 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-calling.txt 2024-04-24T23:16:17.532 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-api.txt 2024-04-24T23:16:17.533 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-decoder-regs.txt 2024-04-24T23:16:17.534 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-dma.txt 2024-04-24T23:16:17.535 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-encoder-api.txt 2024-04-24T23:16:17.536 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-memory.txt 2024-04-24T23:16:17.537 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-osd-api.txt 2024-04-24T23:16:17.538 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx2341x/fw-upload.txt 2024-04-24T23:16:17.539 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx88/ 2024-04-24T23:16:17.540 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/cx88/hauppauge-wintv-cx88-ir.txt 2024-04-24T23:16:17.541 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/extract_xc3028.pl 2024-04-24T23:16:17.542 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/fimc.txt 2024-04-24T23:16:17.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/gspca.txt 2024-04-24T23:16:17.543 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/hauppauge-wintv-cx88-ir.txt 2024-04-24T23:16:17.544 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/lifeview.txt 2024-04-24T23:16:17.545 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/meye.txt 2024-04-24T23:16:17.546 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/not-in-cx2388x-datasheet.txt 2024-04-24T23:16:17.547 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/omap3isp.txt 2024-04-24T23:16:17.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/omap4_camera.txt 2024-04-24T23:16:17.548 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/pxa_camera.txt 2024-04-24T23:16:17.549 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/radiotrack.txt 2024-04-24T23:16:17.550 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/sh_mobile_ceu_camera.txt 2024-04-24T23:16:17.551 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/si470x.txt 2024-04-24T23:16:17.552 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/si4713.txt 2024-04-24T23:16:17.553 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/si476x.txt 2024-04-24T23:16:17.554 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/soc-camera.txt 2024-04-24T23:16:17.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/uvcvideo.txt 2024-04-24T23:16:17.555 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/v4l2-controls.txt 2024-04-24T23:16:17.556 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/v4l2-framework.txt 2024-04-24T23:16:17.557 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/v4l2-pci-skeleton.c 2024-04-24T23:16:17.558 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/videobuf 2024-04-24T23:16:17.559 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/vivid.txt 2024-04-24T23:16:17.560 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/video4linux/zr364xx.txt 2024-04-24T23:16:17.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/ 2024-04-24T23:16:17.561 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/00-INDEX 2024-04-24T23:16:17.562 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/ 2024-04-24T23:16:17.563 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/00-INDEX 2024-04-24T23:16:17.564 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/api.txt 2024-04-24T23:16:17.566 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/cpuid.txt 2024-04-24T23:16:17.567 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/ 2024-04-24T23:16:17.568 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/README 2024-04-24T23:16:17.569 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/arm-vgic.txt 2024-04-24T23:16:17.570 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/mpic.txt 2024-04-24T23:16:17.571 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/s390_flic.txt 2024-04-24T23:16:17.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vfio.txt 2024-04-24T23:16:17.572 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/vm.txt 2024-04-24T23:16:17.574 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/devices/xics.txt 2024-04-24T23:16:17.575 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/hypercalls.txt 2024-04-24T23:16:17.576 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/locking.txt 2024-04-24T23:16:17.577 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/mmu.txt 2024-04-24T23:16:17.578 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/msr.txt 2024-04-24T23:16:17.579 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/nested-vmx.txt 2024-04-24T23:16:17.580 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/ppc-pv.txt 2024-04-24T23:16:17.581 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/review-checklist.txt 2024-04-24T23:16:17.582 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/s390-diag.txt 2024-04-24T23:16:17.583 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/kvm/timekeeping.txt 2024-04-24T23:16:17.584 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/paravirt_ops.txt 2024-04-24T23:16:17.585 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/uml/ 2024-04-24T23:16:17.586 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/virtual/uml/UserModeLinux-HOWTO.txt 2024-04-24T23:16:17.588 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/ 2024-04-24T23:16:17.589 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/.gitignore 2024-04-24T23:16:17.590 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/00-INDEX 2024-04-24T23:16:17.591 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/active_mm.txt 2024-04-24T23:16:17.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/balance 2024-04-24T23:16:17.592 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/cleancache.txt 2024-04-24T23:16:17.593 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/frontswap.txt 2024-04-24T23:16:17.595 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/highmem.txt 2024-04-24T23:16:17.596 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/hugetlbpage.txt 2024-04-24T23:16:17.597 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/hwpoison.txt 2024-04-24T23:16:17.598 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/ksm.txt 2024-04-24T23:16:17.599 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/numa 2024-04-24T23:16:17.600 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/numa_memory_policy.txt 2024-04-24T23:16:17.601 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/overcommit-accounting 2024-04-24T23:16:17.602 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/page_migration 2024-04-24T23:16:17.603 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/page_owner.txt 2024-04-24T23:16:17.604 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/pagemap.txt 2024-04-24T23:16:17.605 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/remap_file_pages.txt 2024-04-24T23:16:17.606 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/slub.txt 2024-04-24T23:16:17.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/soft-dirty.txt 2024-04-24T23:16:17.608 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/split_page_table_lock 2024-04-24T23:16:17.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/transhuge.txt 2024-04-24T23:16:17.610 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/unevictable-lru.txt 2024-04-24T23:16:17.612 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vm/zswap.txt 2024-04-24T23:16:17.613 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/vme_api.txt 2024-04-24T23:16:17.614 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/volatile-considered-harmful.txt 2024-04-24T23:16:17.615 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/ 2024-04-24T23:16:17.616 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/00-INDEX 2024-04-24T23:16:17.617 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/ 2024-04-24T23:16:17.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/00-INDEX 2024-04-24T23:16:17.618 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/ds2482 2024-04-24T23:16:17.619 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/ds2490 2024-04-24T23:16:17.620 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/mxc-w1 2024-04-24T23:16:17.621 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/omap-hdq 2024-04-24T23:16:17.622 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/masters/w1-gpio 2024-04-24T23:16:17.623 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/ 2024-04-24T23:16:17.624 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/00-INDEX 2024-04-24T23:16:17.625 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2406 2024-04-24T23:16:17.626 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds2423 2024-04-24T23:16:17.627 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/w1_ds28e04 2024-04-24T23:16:17.628 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/slaves/w1_therm 2024-04-24T23:16:17.629 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/w1.generic 2024-04-24T23:16:17.630 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/w1/w1.netlink 2024-04-24T23:16:17.631 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/ 2024-04-24T23:16:17.632 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/Makefile 2024-04-24T23:16:17.633 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/convert_drivers_to_kernel_api.txt 2024-04-24T23:16:17.634 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/hpwdt.txt 2024-04-24T23:16:17.635 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/pcwd-watchdog.txt 2024-04-24T23:16:17.636 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/ 2024-04-24T23:16:17.637 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/.gitignore 2024-04-24T23:16:17.638 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/Makefile 2024-04-24T23:16:17.639 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-simple.c 2024-04-24T23:16:17.640 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/src/watchdog-test.c 2024-04-24T23:16:17.641 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/watchdog-api.txt 2024-04-24T23:16:17.642 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/watchdog-kernel-api.txt 2024-04-24T23:16:17.643 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/watchdog-parameters.txt 2024-04-24T23:16:17.644 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/watchdog/wdt.txt 2024-04-24T23:16:17.645 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/wimax/ 2024-04-24T23:16:17.646 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/wimax/README.i2400m 2024-04-24T23:16:17.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/wimax/README.wimax 2024-04-24T23:16:17.647 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/workqueue.txt 2024-04-24T23:16:17.649 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/ 2024-04-24T23:16:17.650 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/00-INDEX 2024-04-24T23:16:17.650 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/boot.txt 2024-04-24T23:16:17.651 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/early-microcode.txt 2024-04-24T23:16:17.652 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/earlyprintk.txt 2024-04-24T23:16:17.653 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/entry_64.txt 2024-04-24T23:16:17.654 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/exception-tables.txt 2024-04-24T23:16:17.655 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/i386/ 2024-04-24T23:16:17.656 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/i386/IO-APIC.txt 2024-04-24T23:16:17.657 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/intel_mpx.txt 2024-04-24T23:16:17.658 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/mtrr.txt 2024-04-24T23:16:17.659 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/pat.txt 2024-04-24T23:16:17.660 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/tlb.txt 2024-04-24T23:16:17.661 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/usb-legacy-support.txt 2024-04-24T23:16:17.662 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/ 2024-04-24T23:16:17.663 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/00-INDEX 2024-04-24T23:16:17.664 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/boot-options.txt 2024-04-24T23:16:17.665 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/cpu-hotplug-spec 2024-04-24T23:16:17.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/fake-numa-for-cpusets 2024-04-24T23:16:17.666 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/kernel-stacks 2024-04-24T23:16:17.668 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/machinecheck 2024-04-24T23:16:17.669 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/mm.txt 2024-04-24T23:16:17.670 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/x86_64/uefi.txt 2024-04-24T23:16:17.670 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/x86/zero-page.txt 2024-04-24T23:16:17.671 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xillybus.txt 2024-04-24T23:16:17.672 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xtensa/ 2024-04-24T23:16:17.673 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xtensa/atomctl.txt 2024-04-24T23:16:17.674 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xtensa/mmu.txt 2024-04-24T23:16:17.675 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/xz.txt 2024-04-24T23:16:17.676 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/ 2024-04-24T23:16:17.677 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/CodingStyle 2024-04-24T23:16:17.678 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/HOWTO 2024-04-24T23:16:17.679 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/IRQ.txt 2024-04-24T23:16:17.680 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/SecurityBugs 2024-04-24T23:16:17.681 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingDrivers 2024-04-24T23:16:17.682 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/SubmittingPatches 2024-04-24T23:16:17.683 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm/ 2024-04-24T23:16:17.684 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm/Booting 2024-04-24T23:16:17.685 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm/kernel_user_helpers.txt 2024-04-24T23:16:17.686 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm64/ 2024-04-24T23:16:17.687 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm64/booting.txt 2024-04-24T23:16:17.688 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm64/memory.txt 2024-04-24T23:16:17.689 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/arm64/tagged-pointers.txt 2024-04-24T23:16:17.690 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/basic_profiling.txt 2024-04-24T23:16:17.691 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/email-clients.txt 2024-04-24T23:16:17.692 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/ 2024-04-24T23:16:17.693 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/filesystems/sysfs.txt 2024-04-24T23:16:17.694 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/gpio.txt 2024-04-24T23:16:17.695 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/io_ordering.txt 2024-04-24T23:16:17.920 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/magic-number.txt 2024-04-24T23:16:17.921 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/oops-tracing.txt 2024-04-24T23:16:17.922 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/sparse.txt 2024-04-24T23:16:17.923 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/stable_api_nonsense.txt 2024-04-24T23:16:17.924 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/stable_kernel_rules.txt 2024-04-24T23:16:17.926 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/ 2024-04-24T23:16:17.927 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/omap3isp.txt 2024-04-24T23:16:17.928 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/video4linux/v4l2-framework.txt 2024-04-24T23:16:17.929 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zh_CN/volatile-considered-harmful.txt 2024-04-24T23:16:17.930 INFO:tasks.workunit.client.0.smithi112.stdout:linux-4.0.5/Documentation/zorro.txt 2024-04-24T23:16:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:18 smithi112 ceph-mon[29186]: pgmap v409: 129 pgs: 129 active+clean; 75 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.2 MiB/s wr, 53 op/s 2024-04-24T23:16:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:18 smithi150 ceph-mon[32247]: pgmap v409: 129 pgs: 129 active+clean; 75 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.2 MiB/s wr, 53 op/s 2024-04-24T23:16:18.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:18 smithi187 ceph-mon[33292]: pgmap v409: 129 pgs: 129 active+clean; 75 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 6.2 MiB/s wr, 53 op/s 2024-04-24T23:16:20.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:20 smithi112 ceph-mon[29186]: pgmap v410: 129 pgs: 129 active+clean; 88 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 7.3 MiB/s wr, 56 op/s 2024-04-24T23:16:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:20 smithi150 ceph-mon[32247]: pgmap v410: 129 pgs: 129 active+clean; 88 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 7.3 MiB/s wr, 56 op/s 2024-04-24T23:16:20.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:20 smithi187 ceph-mon[33292]: pgmap v410: 129 pgs: 129 active+clean; 88 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 7.3 MiB/s wr, 56 op/s 2024-04-24T23:16:22.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:22 smithi112 ceph-mon[29186]: pgmap v411: 129 pgs: 129 active+clean; 91 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 7.5 MiB/s wr, 58 op/s 2024-04-24T23:16:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:22 smithi150 ceph-mon[32247]: pgmap v411: 129 pgs: 129 active+clean; 91 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 7.5 MiB/s wr, 58 op/s 2024-04-24T23:16:22.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:22 smithi187 ceph-mon[33292]: pgmap v411: 129 pgs: 129 active+clean; 91 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 7.5 MiB/s wr, 58 op/s 2024-04-24T23:16:22.761 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar cf doc.tar linux-4.0.5 2024-04-24T23:16:23.731 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar xf doc.tar 2024-04-24T23:16:24.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:24 smithi112 ceph-mon[29186]: pgmap v412: 129 pgs: 129 active+clean; 92 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 7.7 MiB/s wr, 58 op/s 2024-04-24T23:16:24.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:24 smithi150 ceph-mon[32247]: pgmap v412: 129 pgs: 129 active+clean; 92 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 7.7 MiB/s wr, 58 op/s 2024-04-24T23:16:24.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:24 smithi187 ceph-mon[33292]: pgmap v412: 129 pgs: 129 active+clean; 92 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 7.7 MiB/s wr, 58 op/s 2024-04-24T23:16:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:26 smithi112 ceph-mon[29186]: pgmap v413: 129 pgs: 129 active+clean; 92 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 7.7 MiB/s wr, 63 op/s 2024-04-24T23:16:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:26 smithi150 ceph-mon[32247]: pgmap v413: 129 pgs: 129 active+clean; 92 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 7.7 MiB/s wr, 63 op/s 2024-04-24T23:16:26.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:26 smithi187 ceph-mon[33292]: pgmap v413: 129 pgs: 129 active+clean; 92 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 7.7 MiB/s wr, 63 op/s 2024-04-24T23:16:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:28 smithi150 ceph-mon[32247]: pgmap v414: 129 pgs: 129 active+clean; 95 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 6.2 MiB/s wr, 77 op/s 2024-04-24T23:16:28.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:28 smithi187 ceph-mon[33292]: pgmap v414: 129 pgs: 129 active+clean; 95 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 6.2 MiB/s wr, 77 op/s 2024-04-24T23:16:28.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:28 smithi112 ceph-mon[29186]: pgmap v414: 129 pgs: 129 active+clean; 95 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 6.2 MiB/s wr, 77 op/s 2024-04-24T23:16:30.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:30 smithi150 ceph-mon[32247]: pgmap v415: 129 pgs: 129 active+clean; 100 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 56 op/s 2024-04-24T23:16:30.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:30 smithi187 ceph-mon[33292]: pgmap v415: 129 pgs: 129 active+clean; 100 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 56 op/s 2024-04-24T23:16:30.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:30 smithi112 ceph-mon[29186]: pgmap v415: 129 pgs: 129 active+clean; 100 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 56 op/s 2024-04-24T23:16:31.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:16:31.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:16:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.316+0000 7f70efbd5640 1 -- 172.21.15.112:0/1459749613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 msgr2=0x7f70e81045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.316+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1459749613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 0x7f70e81045f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009ca0 tx=0x7f70e402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 -- 172.21.15.112:0/1459749613 shutdown_connections 2024-04-24T23:16:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1459749613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70e8107460 0x7f70e81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1459749613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e8104b30 0x7f70e8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1459749613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 0x7f70e81045f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 -- 172.21.15.112:0/1459749613 >> 172.21.15.112:0/1459749613 conn(0x7f70e80fbbf0 msgr2=0x7f70e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 -- 172.21.15.112:0/1459749613 shutdown_connections 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 -- 172.21.15.112:0/1459749613 wait complete. 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 Processor -- start 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 -- start start 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 0x7f70e8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e8104b30 0x7f70e81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70e8107460 0x7f70e819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70e810d530 con 0x7f70e8102200 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70e810d3b0 con 0x7f70e8107460 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.317+0000 7f70efbd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70e810d6b0 con 0x7f70e8104b30 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 0x7f70e8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 0x7f70e8197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57374/0 (socket says 172.21.15.112:57374) 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 -- 172.21.15.112:0/1374356433 learned_addr learned my addr 172.21.15.112:0/1374356433 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed170640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e8104b30 0x7f70e81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ee172640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70e8107460 0x7f70e819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 -- 172.21.15.112:0/1374356433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e8104b30 msgr2=0x7f70e81980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e8104b30 0x7f70e81980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 -- 172.21.15.112:0/1374356433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70e8107460 msgr2=0x7f70e819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70e8107460 0x7f70e819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 -- 172.21.15.112:0/1374356433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e819f790 con 0x7f70e8102200 2024-04-24T23:16:31.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ee172640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70e8107460 0x7f70e819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70ed971640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 0x7f70e8197b70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f70e4006010 tx=0x7f70e4031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70deffd640 1 -- 172.21.15.112:0/1374356433 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e402fb20 con 0x7f70e8102200 2024-04-24T23:16:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70efbd5640 1 -- 172.21.15.112:0/1374356433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e819fa20 con 0x7f70e8102200 2024-04-24T23:16:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70deffd640 1 -- 172.21.15.112:0/1374356433 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f70e402fcc0 con 0x7f70e8102200 2024-04-24T23:16:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70deffd640 1 -- 172.21.15.112:0/1374356433 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4046940 con 0x7f70e8102200 2024-04-24T23:16:31.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.318+0000 7f70efbd5640 1 -- 172.21.15.112:0/1374356433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e819ff70 con 0x7f70e8102200 2024-04-24T23:16:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.320+0000 7f70deffd640 1 -- 172.21.15.112:0/1374356433 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f70e4030be0 con 0x7f70e8102200 2024-04-24T23:16:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70f29b6740 1 Processor -- start 2024-04-24T23:16:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70f29b6740 1 -- start start 2024-04-24T23:16:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f58660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58c30 0x563963f5d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f60230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70f29b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563963ebc6b0 con 0x563963f58c30 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70f29b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563963e64880 con 0x563963f5dda0 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70f29b6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563963eb8670 con 0x563963f58240 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70deffd640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70d40a2b90 0x7f70d40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70ed971640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58c30 0x563963f5d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70ed170640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f60230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70ed170640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f60230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39124/0 (socket says 172.21.15.112:39124) 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.321+0000 7f70ed170640 1 -- 172.21.15.112:0/1239785482 learned_addr learned my addr 172.21.15.112:0/1239785482 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ee172640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70d40a2b90 0x7f70d40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ee172640 1 --2- 172.21.15.112:0/1239785482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f58660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70deffd640 1 -- 172.21.15.112:0/1374356433 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f70e40f0970 con 0x7f70e8102200 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ed170640 1 -- 172.21.15.112:0/1239785482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 msgr2=0x563963f58660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ed170640 1 --2- 172.21.15.112:0/1239785482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f58660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ed170640 1 -- 172.21.15.112:0/1239785482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58c30 msgr2=0x563963f5d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ed170640 1 --2- 172.21.15.112:0/1239785482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58c30 0x563963f5d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ed170640 1 -- 172.21.15.112:0/1239785482 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563963c9a7d0 con 0x563963f5dda0 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ee172640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70d40a2b90 0x7f70d40a5050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x563963f59490 tx=0x7f70e0009d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70ed170640 1 --2- 172.21.15.112:0/1239785482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f60230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70d800ac70 tx=0x7f70d800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c7e0c0f7c9c2c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70d3fff640 1 -- 172.21.15.112:0/1239785482 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d801d070 con 0x563963f5dda0 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70d3fff640 1 -- 172.21.15.112:0/1239785482 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f70d8018b40 con 0x563963f5dda0 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.322+0000 7f70d3fff640 1 -- 172.21.15.112:0/1239785482 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d8010ca0 con 0x563963f5dda0 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 -- 172.21.15.112:0/1239785482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 msgr2=0x563963f60230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 --2- 172.21.15.112:0/1239785482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f60230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f70d800ac70 tx=0x7f70d800f790 comp rx=0 tx=0).stop 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 -- 172.21.15.112:0/1239785482 shutdown_connections 2024-04-24T23:16:31.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 --2- 172.21.15.112:0/1239785482 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f60230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 --2- 172.21.15.112:0/1239785482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58c30 0x563963f5d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 --2- 172.21.15.112:0/1239785482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f58660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 -- 172.21.15.112:0/1239785482 >> 172.21.15.112:0/1239785482 conn(0x563963f538b0 msgr2=0x563963f53cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 -- 172.21.15.112:0/1239785482 shutdown_connections 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 -- 172.21.15.112:0/1239785482 wait complete. 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 -- start start 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.323+0000 7f70f29b6740 1 Processor -- start 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58240 0x563963f62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58c30 0x563963f62ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f69be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70f29b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563963f60d50 con 0x563963f58240 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70f29b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563963e77f50 con 0x563963f5dda0 2024-04-24T23:16:31.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70f29b6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563963f60ed0 con 0x563963f58c30 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70f29b6740 10 client.0 _ll_register_callbacks cb 0x563963eb3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58240 0x563963f62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58240 0x563963f62660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57390/0 (socket says 172.21.15.112:57390) 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 -- 172.21.15.112:0/2293778664 learned_addr learned my addr 172.21.15.112:0/2293778664 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed170640 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58c30 0x563963f62ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ee172640 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f69be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 -- 172.21.15.112:0/2293778664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58c30 msgr2=0x563963f62ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58c30 0x563963f62ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 -- 172.21.15.112:0/2293778664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 msgr2=0x563963f69be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f69be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 -- 172.21.15.112:0/2293778664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563963bd03f0 con 0x563963f58240 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed170640 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58c30 0x563963f62ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:31.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70ed971640 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58240 0x563963f62660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f70e40bcae0 tx=0x7f70e402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208fdbf687255d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70d17fa640 1 -- 172.21.15.112:0/2293778664 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4004d40 con 0x563963f58240 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70d17fa640 1 -- 172.21.15.112:0/2293778664 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f70e40bc230 con 0x563963f58240 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.324+0000 7f70f29b6740 1 -- 172.21.15.112:0/2293778664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d8018b40 con 0x563963f58240 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70d17fa640 1 -- 172.21.15.112:0/2293778664 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70e40bd4e0 con 0x563963f58240 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70f29b6740 10 client.16005 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70f29b6740 1 -- 172.21.15.112:0/2293778664 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563963b900f0 con 0x563963f58240 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70d17fa640 1 -- 172.21.15.112:0/2293778664 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f70d8018b40 con 0x563963f58240 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70d17fa640 1 -- 172.21.15.112:0/2293778664 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70e404e030 con 0x563963f58240 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70f29b6740 10 client.16005 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70f29b6740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70f29b6740 20 client.16005 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70f29b6740 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563963f6e3f0 0x563963f70880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70f29b6740 4 client.16005 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70f29b6740 1 -- 172.21.15.112:0/2293778664 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563963ec9bf0 con 0x563963f6e3f0 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.325+0000 7f70ed170640 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563963f6e3f0 0x563963f70880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.326+0000 7f70d17fa640 10 client.16005 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.326+0000 7f70ed170640 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563963f6e3f0 0x563963f70880 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6149cfbce7948c server_cookie=483bbf9cf4540b13 in_seq=0 out_seq=0 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.326+0000 7f70d17fa640 1 -- 172.21.15.112:0/2293778664 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563963ec9bf0 con 0x563963f6e3f0 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.326+0000 7f70d17fa640 10 client.16005 handle_command_reply: tid=0 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.326+0000 7f70f29b6740 1 client.16005 shutdown 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 -- 172.21.15.112:0/2293778664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58240 msgr2=0x563963f62660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58240 0x563963f62660 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f70e40bcae0 tx=0x7f70e402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 -- 172.21.15.112:0/2293778664 shutdown_connections 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563963f6e3f0 0x563963f70880 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f5dda0 0x563963f69be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58c30 0x563963f62ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70d17fa640 0 client.16005 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:31.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 --2- 172.21.15.112:0/2293778664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f58240 0x563963f62660 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 -- 172.21.15.112:0/2293778664 >> 172.21.15.112:0/2293778664 conn(0x563963f538b0 msgr2=0x563963f52af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 -- 172.21.15.112:0/2293778664 shutdown_connections 2024-04-24T23:16:31.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 1 -- 172.21.15.112:0/2293778664 wait complete. 2024-04-24T23:16:31.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.327+0000 7f70f29b6740 20 client.16005 trim_cache size 0 max 16384 2024-04-24T23:16:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70f29b6740 1 Processor -- start 2024-04-24T23:16:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70f29b6740 1 -- start start 2024-04-24T23:16:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f53c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f69ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f62980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70f29b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81aa4f0 con 0x563963f5dda0 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70f29b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81ac030 con 0x563963f58c30 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70f29b6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81aabc0 con 0x563963f58240 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70ee172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f62980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70ee172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f62980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57398/0 (socket says 172.21.15.112:57398) 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70ed170640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f69ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.344+0000 7f70ee172640 1 -- 172.21.15.112:0/1980368481 learned_addr learned my addr 172.21.15.112:0/1980368481 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70ed971640 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f53c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70ee172640 1 -- 172.21.15.112:0/1980368481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 msgr2=0x563963f53c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70ee172640 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f53c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70ee172640 1 -- 172.21.15.112:0/1980368481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 msgr2=0x563963f69ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70ee172640 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f69ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70ee172640 1 -- 172.21.15.112:0/1980368481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563963f53450 con 0x563963f5dda0 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70ed170640 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f69ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70ee172640 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f62980 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f70e0004170 tx=0x7f70e0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75e0ade80c1b8af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70d37fe640 1 -- 172.21.15.112:0/1980368481 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e002d070 con 0x563963f5dda0 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70d37fe640 1 -- 172.21.15.112:0/1980368481 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f70e00035f0 con 0x563963f5dda0 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70d37fe640 1 -- 172.21.15.112:0/1980368481 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0028410 con 0x563963f5dda0 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70f29b6740 1 -- 172.21.15.112:0/1980368481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 msgr2=0x563963f62980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.345+0000 7f70f29b6740 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f62980 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f70e0004170 tx=0x7f70e0005f90 comp rx=0 tx=0).stop 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 -- 172.21.15.112:0/1980368481 shutdown_connections 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f62980 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f69ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 --2- 172.21.15.112:0/1980368481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f53c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 -- 172.21.15.112:0/1980368481 >> 172.21.15.112:0/1980368481 conn(0x563963f538b0 msgr2=0x563963f62270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 -- 172.21.15.112:0/1980368481 shutdown_connections 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 -- 172.21.15.112:0/1980368481 wait complete. 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 -- start start 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 Processor -- start 2024-04-24T23:16:31.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f67720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81ab960 con 0x563963f5dda0 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81ab290 con 0x563963f58c30 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70e81a84d0 con 0x563963f58240 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70f29b6740 10 client.0 _ll_register_callbacks cb 0x7f70e4009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70ee172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70ee172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57410/0 (socket says 172.21.15.112:57410) 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70ee172640 1 -- 172.21.15.112:0/3376744158 learned_addr learned my addr 172.21.15.112:0/3376744158 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70ed971640 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.346+0000 7f70ed170640 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f67720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70ee172640 1 -- 172.21.15.112:0/3376744158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 msgr2=0x563963f671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70ee172640 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70ee172640 1 -- 172.21.15.112:0/3376744158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 msgr2=0x563963f67720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70ee172640 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f67720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70ee172640 1 -- 172.21.15.112:0/3376744158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563963f68cd0 con 0x563963f5dda0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70ee172640 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f686e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009250 tx=0x7f70e00039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ebedd552ea7978 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70d17fa640 1 -- 172.21.15.112:0/3376744158 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e002d030 con 0x563963f5dda0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70d17fa640 1 -- 172.21.15.112:0/3376744158 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f70e0025040 con 0x563963f5dda0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 1 -- 172.21.15.112:0/3376744158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70e00035f0 con 0x563963f5dda0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70d17fa640 1 -- 172.21.15.112:0/3376744158 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f70e003a050 con 0x563963f5dda0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 10 client.16011 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 1 -- 172.21.15.112:0/3376744158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563963fd4a00 con 0x563963f5dda0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70d17fa640 1 -- 172.21.15.112:0/3376744158 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f70e00035f0 con 0x563963f5dda0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70d17fa640 1 -- 172.21.15.112:0/3376744158 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70e0004c20 con 0x563963f5dda0 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 10 client.16011 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:31.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 20 client.16011 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:31.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563963fd5270 0x563963fd7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 4 client.16011 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:16:31.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.347+0000 7f70f29b6740 1 -- 172.21.15.112:0/3376744158 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563963ec3f40 con 0x563963fd5270 2024-04-24T23:16:31.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.348+0000 7f70ed971640 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563963fd5270 0x563963fd7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.348+0000 7f70d17fa640 10 client.16011 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:31.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.348+0000 7f70ed971640 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563963fd5270 0x563963fd7690 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a2d2def4f9c8b3 server_cookie=f0a1b86dd02362c3 in_seq=0 out_seq=0 2024-04-24T23:16:31.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70d17fa640 1 -- 172.21.15.112:0/3376744158 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563963ec3f40 con 0x563963fd5270 2024-04-24T23:16:31.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70d17fa640 10 client.16011 handle_command_reply: tid=0 2024-04-24T23:16:31.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 client.16011 shutdown 2024-04-24T23:16:31.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 -- 172.21.15.112:0/3376744158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 msgr2=0x563963f686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f686e0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009250 tx=0x7f70e00039d0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 -- 172.21.15.112:0/3376744158 shutdown_connections 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563963fd5270 0x563963fd7690 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563963f5dda0 0x563963f686e0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563963f58c30 0x563963f67720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70d17fa640 0 client.16011 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 --2- 172.21.15.112:0/3376744158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563963f58240 0x563963f671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.349+0000 7f70f29b6740 1 -- 172.21.15.112:0/3376744158 >> 172.21.15.112:0/3376744158 conn(0x563963f538b0 msgr2=0x563963f56860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70f29b6740 1 -- 172.21.15.112:0/3376744158 shutdown_connections 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70f29b6740 1 -- 172.21.15.112:0/3376744158 wait complete. 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70f29b6740 20 client.16011 trim_cache size 0 max 16384 2024-04-24T23:16:31.357 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:16:31.358 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-04-24T23:16:31.358 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-04-24T23:16:31.358 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:16:31.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 -- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70d40a2b90 msgr2=0x7f70d40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70d40a2b90 0x7f70d40a5050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x563963f59490 tx=0x7f70e0009d00 comp rx=0 tx=0).stop 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 -- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 msgr2=0x7f70e8197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 0x7f70e8197b70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f70e4006010 tx=0x7f70e4031ec0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 -- 172.21.15.112:0/1374356433 shutdown_connections 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f70d40a2b90 0x7f70d40a5050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70e8107460 0x7f70e819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70e8104b30 0x7f70e81980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 --2- 172.21.15.112:0/1374356433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70e8102200 0x7f70e8197b70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 -- 172.21.15.112:0/1374356433 >> 172.21.15.112:0/1374356433 conn(0x7f70e80fbbf0 msgr2=0x7f70e8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 -- 172.21.15.112:0/1374356433 shutdown_connections 2024-04-24T23:16:31.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.350+0000 7f70efbd5640 1 -- 172.21.15.112:0/1374356433 wait complete. 2024-04-24T23:16:31.362 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:16:31.363 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:16:31.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 -- 172.21.15.112:0/3819775978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690102230 msgr2=0x7f1690108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 --2- 172.21.15.112:0/3819775978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690102230 0x7f1690108ac0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1688009ca0 tx=0x7f168802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 -- 172.21.15.112:0/3819775978 shutdown_connections 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 --2- 172.21.15.112:0/3819775978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690102230 0x7f1690108ac0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 --2- 172.21.15.112:0/3819775978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1690101870 0x7f1690101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 --2- 172.21.15.112:0/3819775978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690100680 0x7f1690100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 -- 172.21.15.112:0/3819775978 >> 172.21.15.112:0/3819775978 conn(0x7f16900fbbf0 msgr2=0x7f16900fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 -- 172.21.15.112:0/3819775978 shutdown_connections 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.464+0000 7f1697e88640 1 -- 172.21.15.112:0/3819775978 wait complete. 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 Processor -- start 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 -- start start 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1690100680 0x7f1690069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690101870 0x7f169019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690102230 0x7f169019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c430 con 0x7f1690101870 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c2b0 con 0x7f1690102230 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f169010c5b0 con 0x7f1690100680 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695c24640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1690100680 0x7f1690069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695423640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690101870 0x7f169019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695c24640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1690100680 0x7f1690069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41930/0 (socket says 172.21.15.112:41930) 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695c24640 1 -- 172.21.15.112:0/2797574602 learned_addr learned my addr 172.21.15.112:0/2797574602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1696425640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690102230 0x7f169019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695423640 1 -- 172.21.15.112:0/2797574602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1690100680 msgr2=0x7f1690069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695423640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1690100680 0x7f1690069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695423640 1 -- 172.21.15.112:0/2797574602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690102230 msgr2=0x7f169019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695423640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690102230 0x7f169019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695423640 1 -- 172.21.15.112:0/2797574602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169019f760 con 0x7f1690101870 2024-04-24T23:16:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1695423640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690101870 0x7f169019c830 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f168000c970 tx=0x7f168000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1686ffd640 1 -- 172.21.15.112:0/2797574602 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680007bb0 con 0x7f1690101870 2024-04-24T23:16:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.465+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169019f9f0 con 0x7f1690101870 2024-04-24T23:16:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.466+0000 7f1686ffd640 1 -- 172.21.15.112:0/2797574602 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f1680007d50 con 0x7f1690101870 2024-04-24T23:16:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.466+0000 7f1686ffd640 1 -- 172.21.15.112:0/2797574602 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168000f450 con 0x7f1690101870 2024-04-24T23:16:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.466+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169019ff40 con 0x7f1690101870 2024-04-24T23:16:31.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.467+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16901078d0 con 0x7f1690101870 2024-04-24T23:16:31.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.468+0000 7f1686ffd640 1 -- 172.21.15.112:0/2797574602 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1680004fe0 con 0x7f1690101870 2024-04-24T23:16:31.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.468+0000 7f1686ffd640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f167c0a2b90 0x7f167c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.468+0000 7f1686ffd640 1 -- 172.21.15.112:0/2797574602 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f16800cf3e0 con 0x7f1690101870 2024-04-24T23:16:31.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.468+0000 7f1695c24640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f167c0a2b90 0x7f167c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.469+0000 7f1695c24640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f167c0a2b90 0x7f167c0a5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f168c0058c0 tx=0x7f168c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.472+0000 7f1686ffd640 1 -- 172.21.15.112:0/2797574602 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1680014030 con 0x7f1690101870 2024-04-24T23:16:31.704 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.702+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f169002d170 con 0x7f1690101870 2024-04-24T23:16:31.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.703+0000 7f1686ffd640 1 -- 172.21.15.112:0/2797574602 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11947 (secure 0 0 0) 0x7f168009b900 con 0x7f1690101870 2024-04-24T23:16:31.705 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:16:31.705 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:14:48.128683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24424,24425,24490,24524,24482]},"id":1}]} 2024-04-24T23:16:31.706 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 28 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f167c0a2b90 msgr2=0x7f167c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f167c0a2b90 0x7f167c0a5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f168c0058c0 tx=0x7f168c005f90 comp rx=0 tx=0).stop 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690101870 msgr2=0x7f169019c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690101870 0x7f169019c830 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f168000c970 tx=0x7f168000ce30 comp rx=0 tx=0).stop 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 shutdown_connections 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f167c0a2b90 0x7f167c0a5050 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1690102230 0x7f169019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1690101870 0x7f169019c830 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 --2- 172.21.15.112:0/2797574602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1690100680 0x7f1690069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.705+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 >> 172.21.15.112:0/2797574602 conn(0x7f16900fbbf0 msgr2=0x7f1690104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.706+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 shutdown_connections 2024-04-24T23:16:31.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.706+0000 7f1697e88640 1 -- 172.21.15.112:0/2797574602 wait complete. 2024-04-24T23:16:31.718 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 -- 172.21.15.112:0/3052485695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104101790 msgr2=0x7fb104101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 --2- 172.21.15.112:0/3052485695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104101790 0x7fb104101bb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb100009900 tx=0x7fb10002a410 comp rx=0 tx=0).stop 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 -- 172.21.15.112:0/3052485695 shutdown_connections 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 --2- 172.21.15.112:0/3052485695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb104103340 0x7fb104109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 --2- 172.21.15.112:0/3052485695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104102980 0x7fb104102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 --2- 172.21.15.112:0/3052485695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104101790 0x7fb104101bb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 -- 172.21.15.112:0/3052485695 >> 172.21.15.112:0/3052485695 conn(0x7fb1040fcd20 msgr2=0x7fb1040ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 -- 172.21.15.112:0/3052485695 shutdown_connections 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 -- 172.21.15.112:0/3052485695 wait complete. 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 Processor -- start 2024-04-24T23:16:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.821+0000 7fb10b88c640 1 -- start start 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb10b88c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb104101790 0x7fb1040754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb10b88c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104102980 0x7fb104073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb10b88c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104103340 0x7fb104074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb10b88c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb10410d540 con 0x7fb104103340 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb10b88c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb10410d3c0 con 0x7fb104101790 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb10b88c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb10410d6c0 con 0x7fb104102980 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104103340 0x7fb104074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109628640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb104101790 0x7fb1040754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104103340 0x7fb104074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57442/0 (socket says 172.21.15.112:57442) 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 -- 172.21.15.112:0/1572226666 learned_addr learned my addr 172.21.15.112:0/1572226666 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb108e27640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104102980 0x7fb104073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 -- 172.21.15.112:0/1572226666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104102980 msgr2=0x7fb104073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104102980 0x7fb104073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 -- 172.21.15.112:0/1572226666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb104101790 msgr2=0x7fb1040754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb104101790 0x7fb1040754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 -- 172.21.15.112:0/1572226666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb104074900 con 0x7fb104103340 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109628640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb104101790 0x7fb1040754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb109e29640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104103340 0x7fb104074070 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00eb10 tx=0x7fb0fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb0fa7fc640 1 -- 172.21.15.112:0/1572226666 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc004b20 con 0x7fb104103340 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb10b88c640 1 -- 172.21.15.112:0/1572226666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1041a1cb0 con 0x7fb104103340 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb0fa7fc640 1 -- 172.21.15.112:0/1572226666 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb0fc004cc0 con 0x7fb104103340 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb0fa7fc640 1 -- 172.21.15.112:0/1572226666 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc0107f0 con 0x7fb104103340 2024-04-24T23:16:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.822+0000 7fb10b88c640 1 -- 172.21.15.112:0/1572226666 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1041a21b0 con 0x7fb104103340 2024-04-24T23:16:31.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.825+0000 7fb0fa7fc640 1 -- 172.21.15.112:0/1572226666 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb0fc005020 con 0x7fb104103340 2024-04-24T23:16:31.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.825+0000 7fb10e599740 1 Processor -- start 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb10e599740 1 -- start start 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb0fa7fc640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0f00a2b90 0x7fb0f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb10e599740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c908650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb108e27640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0f00a2b90 0x7fb0f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb10e599740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c908c20 0x56289c90d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb10e599740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c90dd90 0x56289c910220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb10e599740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56289c86c670 con 0x56289c908230 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb10e599740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56289c814970 con 0x56289c90dd90 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb10e599740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56289c868630 con 0x56289c908c20 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb0fa7fc640 1 -- 172.21.15.112:0/1572226666 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb0fc0cfad0 con 0x7fb104103340 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c908650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c908650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57454/0 (socket says 172.21.15.112:57454) 2024-04-24T23:16:31.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 -- 172.21.15.112:0/1901794875 learned_addr learned my addr 172.21.15.112:0/1901794875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb108e27640 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c908c20 0x56289c90d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109e29640 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c90dd90 0x56289c910220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 -- 172.21.15.112:0/1901794875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c908c20 msgr2=0x56289c90d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c908c20 0x56289c90d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 -- 172.21.15.112:0/1901794875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c90dd90 msgr2=0x56289c910220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c90dd90 0x56289c910220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb108e27640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0f00a2b90 0x7fb0f00a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005fd0 tx=0x7fb0f40127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 -- 172.21.15.112:0/1901794875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56289c838400 con 0x56289c908230 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.826+0000 7fb109628640 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c908650 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb100009900 tx=0x7fb100005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d047200a9a4b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/1901794875 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100040070 con 0x56289c908230 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/1901794875 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb100004230 con 0x56289c908230 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb0ef7fe640 1 -- 172.21.15.112:0/1901794875 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100035040 con 0x56289c908230 2024-04-24T23:16:31.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 -- 172.21.15.112:0/1901794875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 msgr2=0x56289c908650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c908650 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb100009900 tx=0x7fb100005d50 comp rx=0 tx=0).stop 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 -- 172.21.15.112:0/1901794875 shutdown_connections 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c90dd90 0x56289c910220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c908c20 0x56289c90d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 --2- 172.21.15.112:0/1901794875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c908650 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 -- 172.21.15.112:0/1901794875 >> 172.21.15.112:0/1901794875 conn(0x56289c9038a0 msgr2=0x56289c903ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 -- 172.21.15.112:0/1901794875 shutdown_connections 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.827+0000 7fb10e599740 1 -- 172.21.15.112:0/1901794875 wait complete. 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 1 -- start start 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 1 Processor -- start 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c912300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c912840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c919880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56289c910ce0 con 0x56289c908230 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56289c828050 con 0x56289c908c20 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56289c910e60 con 0x56289c90dd90 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb109628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c912300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb10e599740 10 client.0 _ll_register_callbacks cb 0x56289c57fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb109628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c912300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57464/0 (socket says 172.21.15.112:57464) 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb109628640 1 -- 172.21.15.112:0/2016826688 learned_addr learned my addr 172.21.15.112:0/2016826688 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.828+0000 7fb108e27640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c912840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb109e29640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c919880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb109628640 1 -- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 msgr2=0x56289c919880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb109628640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c919880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb109628640 1 -- 172.21.15.112:0/2016826688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 msgr2=0x56289c912840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb109628640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c912840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb109628640 1 -- 172.21.15.112:0/2016826688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56289c7f9dc0 con 0x56289c908230 2024-04-24T23:16:31.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb108e27640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c912840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb109e29640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c919880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb109628640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c912300 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb10002a930 tx=0x7fb100009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f9ae62b46b23ea server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb0ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb0ecff9640 1 -- 172.21.15.112:0/2016826688 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10004c030 con 0x56289c908230 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb0ecff9640 1 -- 172.21.15.112:0/2016826688 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb100003800 con 0x56289c908230 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb10e599740 1 -- 172.21.15.112:0/2016826688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb100004230 con 0x56289c908230 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb0ecff9640 1 -- 172.21.15.112:0/2016826688 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb100056050 con 0x56289c908230 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb10e599740 10 client.16029 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb10e599740 1 -- 172.21.15.112:0/2016826688 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56289c5803f0 con 0x56289c908230 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.829+0000 7fb0ecff9640 1 -- 172.21.15.112:0/2016826688 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb100004230 con 0x56289c908230 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb0ecff9640 1 -- 172.21.15.112:0/2016826688 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb100051400 con 0x56289c908230 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb10e599740 10 client.16029 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb10e599740 10 client.16029 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:16:31.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb10e599740 20 client.16029 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb10e599740 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x56289c91dd70 0x56289c920160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb10e599740 4 client.16029 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb10e599740 1 -- 172.21.15.112:0/2016826688 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56289c879bb0 con 0x56289c91dd70 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb108e27640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x56289c91dd70 0x56289c920160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb0ecff9640 10 client.16029 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.830+0000 7fb108e27640 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x56289c91dd70 0x56289c920160 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f00ca380db560ea8 server_cookie=9fbb60690e824a63 in_seq=0 out_seq=0 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb0ecff9640 1 -- 172.21.15.112:0/2016826688 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56289c879bb0 con 0x56289c91dd70 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb0ecff9640 10 client.16029 handle_command_reply: tid=0 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 client.16029 shutdown 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 -- 172.21.15.112:0/2016826688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 msgr2=0x56289c912300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c912300 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb10002a930 tx=0x7fb100009630 comp rx=0 tx=0).stop 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 -- 172.21.15.112:0/2016826688 shutdown_connections 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x56289c91dd70 0x56289c920160 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c919880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c912840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb0ecff9640 0 client.16029 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:16:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 --2- 172.21.15.112:0/2016826688 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c912300 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 -- 172.21.15.112:0/2016826688 >> 172.21.15.112:0/2016826688 conn(0x56289c9038a0 msgr2=0x56289c902b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 -- 172.21.15.112:0/2016826688 shutdown_connections 2024-04-24T23:16:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 1 -- 172.21.15.112:0/2016826688 wait complete. 2024-04-24T23:16:31.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.831+0000 7fb10e599740 20 client.16029 trim_cache size 0 max 16384 2024-04-24T23:16:31.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb10e599740 1 Processor -- start 2024-04-24T23:16:31.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb10e599740 1 -- start start 2024-04-24T23:16:31.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb10e599740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c9022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb10e599740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c9197a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb10e599740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c911920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb10e599740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041ac830 con 0x56289c908230 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb10e599740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041ae370 con 0x56289c908c20 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb10e599740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041acf00 con 0x56289c90dd90 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c9022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c9022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57476/0 (socket says 172.21.15.112:57476) 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109628640 1 -- 172.21.15.112:0/1285663164 learned_addr learned my addr 172.21.15.112:0/1285663164 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb108e27640 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c9197a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109e29640 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c911920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109628640 1 -- 172.21.15.112:0/1285663164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 msgr2=0x56289c911920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109628640 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c911920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109628640 1 -- 172.21.15.112:0/1285663164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 msgr2=0x56289c9197a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109628640 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c9197a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.845+0000 7fb109628640 1 -- 172.21.15.112:0/1285663164 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56289c64a7d0 con 0x56289c908230 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb109628640 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c9022b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb1000096c0 tx=0x7fb100031640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16605173c1864023 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb0eeffd640 1 -- 172.21.15.112:0/1285663164 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10004c040 con 0x56289c908230 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb0eeffd640 1 -- 172.21.15.112:0/1285663164 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb100051c40 con 0x56289c908230 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb0eeffd640 1 -- 172.21.15.112:0/1285663164 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100040070 con 0x56289c908230 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 -- 172.21.15.112:0/1285663164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 msgr2=0x56289c9022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c9022b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb1000096c0 tx=0x7fb100031640 comp rx=0 tx=0).stop 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 -- 172.21.15.112:0/1285663164 shutdown_connections 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c911920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c9197a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 --2- 172.21.15.112:0/1285663164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c9022b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 -- 172.21.15.112:0/1285663164 >> 172.21.15.112:0/1285663164 conn(0x56289c9038a0 msgr2=0x56289c909f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 -- 172.21.15.112:0/1285663164 shutdown_connections 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.846+0000 7fb10e599740 1 -- 172.21.15.112:0/1285663164 wait complete. 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 1 -- start start 2024-04-24T23:16:31.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 1 Processor -- start 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c917240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c917780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c918790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041adca0 con 0x56289c908230 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041ad5d0 con 0x56289c908c20 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041aa810 con 0x56289c90dd90 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb10e599740 10 client.0 _ll_register_callbacks cb 0x7fb0f400a930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb109628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c917240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb109628640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c917240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57482/0 (socket says 172.21.15.112:57482) 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb109628640 1 -- 172.21.15.112:0/1578129712 learned_addr learned my addr 172.21.15.112:0/1578129712 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb108e27640 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c917780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.847+0000 7fb109628640 1 -- 172.21.15.112:0/1578129712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 msgr2=0x56289c918790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb109628640 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c918790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb109628640 1 -- 172.21.15.112:0/1578129712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 msgr2=0x56289c917780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb109628640 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c917780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb109628640 1 -- 172.21.15.112:0/1578129712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56289c918d80 con 0x56289c908230 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb109628640 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c917240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb100009580 tx=0x7fb10002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3e1c9712963954 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb0ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb0ecff9640 1 -- 172.21.15.112:0/1578129712 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100048070 con 0x56289c908230 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb0ecff9640 1 -- 172.21.15.112:0/1578129712 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb1000041f0 con 0x56289c908230 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 1 -- 172.21.15.112:0/1578129712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb100051c40 con 0x56289c908230 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb0ecff9640 1 -- 172.21.15.112:0/1578129712 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb10006c050 con 0x56289c908230 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 10 client.16035 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 1 -- 172.21.15.112:0/1578129712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56289c984440 con 0x56289c908230 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb0ecff9640 1 -- 172.21.15.112:0/1578129712 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb100051c40 con 0x56289c908230 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb0ecff9640 1 -- 172.21.15.112:0/1578129712 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb100005030 con 0x56289c908230 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 10 client.16035 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:31.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 10 client.16035 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 20 client.16035 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x56289c984d30 0x56289c987150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 4 client.16035 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.848+0000 7fb10e599740 1 -- 172.21.15.112:0/1578129712 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56289c873f00 con 0x56289c984d30 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb108e27640 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x56289c984d30 0x56289c987150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb0ecff9640 10 client.16035 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb108e27640 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x56289c984d30 0x56289c987150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=70b72acacfd9fb8b server_cookie=513ab64b8642ae3e in_seq=0 out_seq=0 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb0ecff9640 1 -- 172.21.15.112:0/1578129712 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56289c873f00 con 0x56289c984d30 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb0ecff9640 10 client.16035 handle_command_reply: tid=0 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 client.16035 shutdown 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 -- 172.21.15.112:0/1578129712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 msgr2=0x56289c917240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c917240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb100009580 tx=0x7fb10002a410 comp rx=0 tx=0).stop 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 -- 172.21.15.112:0/1578129712 shutdown_connections 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x56289c984d30 0x56289c987150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56289c90dd90 0x56289c918790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56289c908c20 0x56289c917780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb0ecff9640 0 client.16035 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:16:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 --2- 172.21.15.112:0/1578129712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56289c908230 0x56289c917240 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 -- 172.21.15.112:0/1578129712 >> 172.21.15.112:0/1578129712 conn(0x56289c9038a0 msgr2=0x56289c906780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 -- 172.21.15.112:0/1578129712 shutdown_connections 2024-04-24T23:16:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 1 -- 172.21.15.112:0/1578129712 wait complete. 2024-04-24T23:16:31.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.849+0000 7fb10e599740 20 client.16035 trim_cache size 0 max 16384 2024-04-24T23:16:31.858 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 -- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0f00a2b90 msgr2=0x7fb0f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0f00a2b90 0x7fb0f00a5050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005fd0 tx=0x7fb0f40127c0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 -- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104103340 msgr2=0x7fb104074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104103340 0x7fb104074070 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00eb10 tx=0x7fb0fc00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 -- 172.21.15.112:0/1572226666 shutdown_connections 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb0f00a2b90 0x7fb0f00a5050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb104103340 0x7fb104074070 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104102980 0x7fb104073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 --2- 172.21.15.112:0/1572226666 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb104101790 0x7fb1040754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 -- 172.21.15.112:0/1572226666 >> 172.21.15.112:0/1572226666 conn(0x7fb1040fcd20 msgr2=0x7fb104105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 -- 172.21.15.112:0/1572226666 shutdown_connections 2024-04-24T23:16:31.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.850+0000 7fb10b88c640 1 -- 172.21.15.112:0/1572226666 wait complete. 2024-04-24T23:16:31.862 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T23:16:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 -- 172.21.15.112:0/1838396186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c100680 msgr2=0x7f876c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 --2- 172.21.15.112:0/1838396186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c100680 0x7f876c100aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8764009900 tx=0x7f876402a410 comp rx=0 tx=0).stop 2024-04-24T23:16:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 -- 172.21.15.112:0/1838396186 shutdown_connections 2024-04-24T23:16:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 --2- 172.21.15.112:0/1838396186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f876c102230 0x7f876c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 --2- 172.21.15.112:0/1838396186 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f876c101870 0x7f876c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 --2- 172.21.15.112:0/1838396186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c100680 0x7f876c100aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 -- 172.21.15.112:0/1838396186 >> 172.21.15.112:0/1838396186 conn(0x7f876c0fbbf0 msgr2=0x7f876c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 -- 172.21.15.112:0/1838396186 shutdown_connections 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 -- 172.21.15.112:0/1838396186 wait complete. 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.966+0000 7f8773b70640 1 Processor -- start 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 -- start start 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c100680 0x7f876c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f876c101870 0x7f876c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f876c102230 0x7f876c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10c430 con 0x7f876c101870 2024-04-24T23:16:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10c2b0 con 0x7f876c102230 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f876c10c5b0 con 0x7f876c100680 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f876c101870 0x7f876c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877190c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c100680 0x7f876c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f876c101870 0x7f876c19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57500/0 (socket says 172.21.15.112:57500) 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 -- 172.21.15.112:0/2054317298 learned_addr learned my addr 172.21.15.112:0/2054317298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877210d640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f876c102230 0x7f876c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 -- 172.21.15.112:0/2054317298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c100680 msgr2=0x7f876c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c100680 0x7f876c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 -- 172.21.15.112:0/2054317298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f876c102230 msgr2=0x7f876c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f876c102230 0x7f876c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 -- 172.21.15.112:0/2054317298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c19f820 con 0x7f876c101870 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f877110b640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f876c101870 0x7f876c19c8f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f876000ea60 tx=0x7f876000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f876affd640 1 -- 172.21.15.112:0/2054317298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876000cde0 con 0x7f876c101870 2024-04-24T23:16:31.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 -- 172.21.15.112:0/2054317298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c19fab0 con 0x7f876c101870 2024-04-24T23:16:31.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f876affd640 1 -- 172.21.15.112:0/2054317298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f87600035f0 con 0x7f876c101870 2024-04-24T23:16:31.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f876affd640 1 -- 172.21.15.112:0/2054317298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760010720 con 0x7f876c101870 2024-04-24T23:16:31.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.967+0000 7f8773b70640 1 -- 172.21.15.112:0/2054317298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c1a0000 con 0x7f876c101870 2024-04-24T23:16:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.969+0000 7f876affd640 1 -- 172.21.15.112:0/2054317298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f87600037a0 con 0x7f876c101870 2024-04-24T23:16:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f8776882740 1 Processor -- start 2024-04-24T23:16:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f8776882740 1 -- start start 2024-04-24T23:16:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f8776882740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e6939a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f8776882740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e6939f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f8776882740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f8776882740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e692fe670 con 0x562e6939fd90 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f8776882740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562e692a6970 con 0x562e6939a230 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f8776882740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562e692fa630 con 0x562e6939ac20 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.970+0000 7f876affd640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87540a2b90 0x7f87540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e6939a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877110b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e6939a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39244/0 (socket says 172.21.15.112:39244) 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 -- 172.21.15.112:0/213957089 learned_addr learned my addr 172.21.15.112:0/213957089 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877190c640 1 --2- 172.21.15.112:0/213957089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e6939f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87540a2b90 0x7f87540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f876affd640 1 -- 172.21.15.112:0/2054317298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f87600cf740 con 0x7f876c101870 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 -- 172.21.15.112:0/213957089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 msgr2=0x562e6939f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 --2- 172.21.15.112:0/213957089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e6939f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 -- 172.21.15.112:0/213957089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 msgr2=0x562e693a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 --2- 172.21.15.112:0/213957089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 -- 172.21.15.112:0/213957089 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e692ca400 con 0x562e6939a230 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87540a2b90 0x7f87540a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f875c00fc90 tx=0x7f875c018000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f877210d640 1 --2- 172.21.15.112:0/213957089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e6939a650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f875c011de0 tx=0x7f875c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14cba9cabac24caa server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f8753fff640 1 -- 172.21.15.112:0/213957089 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c022070 con 0x562e6939a230 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f8753fff640 1 -- 172.21.15.112:0/213957089 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f875c011600 con 0x562e6939a230 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.971+0000 7f8753fff640 1 -- 172.21.15.112:0/213957089 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c004290 con 0x562e6939a230 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 -- 172.21.15.112:0/213957089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 msgr2=0x562e6939a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 --2- 172.21.15.112:0/213957089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e6939a650 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f875c011de0 tx=0x7f875c00f790 comp rx=0 tx=0).stop 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 -- 172.21.15.112:0/213957089 shutdown_connections 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 --2- 172.21.15.112:0/213957089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 --2- 172.21.15.112:0/213957089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e6939f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 --2- 172.21.15.112:0/213957089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e6939a650 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 -- 172.21.15.112:0/213957089 >> 172.21.15.112:0/213957089 conn(0x562e693958a0 msgr2=0x562e69395ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 -- 172.21.15.112:0/213957089 shutdown_connections 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.972+0000 7f8776882740 1 -- 172.21.15.112:0/213957089 wait complete. 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 1 -- start start 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 1 Processor -- start 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939a230 0x562e693a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939ac20 0x562e693a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939fd90 0x562e693ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e693a2ce0 con 0x562e6939a230 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x562e692ba050 con 0x562e6939ac20 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562e693a2e60 con 0x562e6939fd90 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f8776882740 10 client.0 _ll_register_callbacks cb 0x562e69011be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877190c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939a230 0x562e693a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877110b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939ac20 0x562e693a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877210d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939fd90 0x562e693ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877110b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939ac20 0x562e693a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39250/0 (socket says 172.21.15.112:39250) 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877190c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939a230 0x562e693a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57520/0 (socket says 172.21.15.112:57520) 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877110b640 1 -- 172.21.15.112:0/841233321 learned_addr learned my addr 172.21.15.112:0/841233321 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877190c640 1 -- 172.21.15.112:0/841233321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939fd90 msgr2=0x562e693ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877190c640 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939fd90 0x562e693ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877190c640 1 -- 172.21.15.112:0/841233321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939ac20 msgr2=0x562e693a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877190c640 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939ac20 0x562e693a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877190c640 1 -- 172.21.15.112:0/841233321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6928bdc0 con 0x562e6939a230 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.973+0000 7f877110b640 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939ac20 0x562e693a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f877190c640 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939a230 0x562e693a42c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8764009230 tx=0x7f8764031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373f94a181705727 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f87517fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f87517fa640 1 -- 172.21.15.112:0/841233321 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764035040 con 0x562e6939a230 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f87517fa640 1 -- 172.21.15.112:0/841233321 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f8764004460 con 0x562e6939a230 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f8776882740 1 -- 172.21.15.112:0/841233321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f875c011600 con 0x562e6939a230 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f87517fa640 1 -- 172.21.15.112:0/841233321 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8764005040 con 0x562e6939a230 2024-04-24T23:16:31.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f8776882740 10 client.16047 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f8776882740 1 -- 172.21.15.112:0/841233321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e690123f0 con 0x562e6939a230 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.974+0000 7f87517fa640 1 -- 172.21.15.112:0/841233321 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f875c011600 con 0x562e6939a230 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f87517fa640 1 -- 172.21.15.112:0/841233321 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f87640051e0 con 0x562e6939a230 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f8776882740 10 client.16047 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f8776882740 10 client.16047 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f8776882740 20 client.16047 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f8776882740 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x562e693b0150 0x562e693b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f8776882740 4 client.16047 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f8776882740 1 -- 172.21.15.112:0/841233321 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e6930bbb0 con 0x562e693b0150 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f877110b640 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x562e693b0150 0x562e693b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f87517fa640 10 client.16047 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:16:31.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.975+0000 7f877110b640 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x562e693b0150 0x562e693b25a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f0819e30d96e9600 server_cookie=e6c7952457064743 in_seq=0 out_seq=0 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.976+0000 7f87517fa640 1 -- 172.21.15.112:0/841233321 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e6930bbb0 con 0x562e693b0150 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.976+0000 7f87517fa640 10 client.16047 handle_command_reply: tid=0 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.976+0000 7f8776882740 1 client.16047 shutdown 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 -- 172.21.15.112:0/841233321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939a230 msgr2=0x562e693a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939a230 0x562e693a42c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8764009230 tx=0x7f8764031970 comp rx=0 tx=0).stop 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 -- 172.21.15.112:0/841233321 shutdown_connections 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x562e693b0150 0x562e693b25a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939fd90 0x562e693ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939ac20 0x562e693a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 --2- 172.21.15.112:0/841233321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939a230 0x562e693a42c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f87517fa640 0 client.16047 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:16:31.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 -- 172.21.15.112:0/841233321 >> 172.21.15.112:0/841233321 conn(0x562e693958a0 msgr2=0x562e69394b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:31.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 -- 172.21.15.112:0/841233321 shutdown_connections 2024-04-24T23:16:31.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 1 -- 172.21.15.112:0/841233321 wait complete. 2024-04-24T23:16:31.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.977+0000 7f8776882740 20 client.16047 trim_cache size 0 max 16384 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f8776882740 1 Processor -- start 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f8776882740 1 -- start start 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f8776882740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e693942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f8776882740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e693b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f8776882740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693b7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f8776882740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f876c1aa570 con 0x562e6939fd90 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f8776882740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f876c1ac0b0 con 0x562e6939a230 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f8776882740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f876c1aac40 con 0x562e6939ac20 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f877210d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:31.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f877210d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57524/0 (socket says 172.21.15.112:57524) 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f877210d640 1 -- 172.21.15.112:0/540373758 learned_addr learned my addr 172.21.15.112:0/540373758 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f877190c640 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e693942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f877110b640 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e693b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f877210d640 1 -- 172.21.15.112:0/540373758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 msgr2=0x562e693b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.990+0000 7f877210d640 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e693b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f877210d640 1 -- 172.21.15.112:0/540373758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 msgr2=0x562e693942b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f877210d640 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e693942b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f877210d640 1 -- 172.21.15.112:0/540373758 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e690dc7d0 con 0x562e6939fd90 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f877110b640 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e693b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f877210d640 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693b7370 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f875c004000 tx=0x7f875c00ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b4479ae4c36bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f87537fe640 1 -- 172.21.15.112:0/540373758 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c022030 con 0x562e6939fd90 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f87537fe640 1 -- 172.21.15.112:0/540373758 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f875c011970 con 0x562e6939fd90 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f87537fe640 1 -- 172.21.15.112:0/540373758 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c0047f0 con 0x562e6939fd90 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 -- 172.21.15.112:0/540373758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 msgr2=0x562e693b7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693b7370 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f875c004000 tx=0x7f875c00ff60 comp rx=0 tx=0).stop 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 -- 172.21.15.112:0/540373758 shutdown_connections 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939fd90 0x562e693b7370 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939ac20 0x562e693b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 --2- 172.21.15.112:0/540373758 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939a230 0x562e693942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 -- 172.21.15.112:0/540373758 >> 172.21.15.112:0/540373758 conn(0x562e693958a0 msgr2=0x562e69398e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 -- 172.21.15.112:0/540373758 shutdown_connections 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.991+0000 7f8776882740 1 -- 172.21.15.112:0/540373758 wait complete. 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 1 -- start start 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 1 Processor -- start 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939a230 0x562e69411690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939ac20 0x562e69411bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939fd90 0x562e69418ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f876c1ab9e0 con 0x562e6939ac20 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f876c1ab310 con 0x562e6939fd90 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f876c1a8550 con 0x562e6939a230 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f8776882740 10 client.0 _ll_register_callbacks cb 0x7f8764009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:32.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f877110b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939ac20 0x562e69411bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f877210d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939fd90 0x562e69418ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f877110b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939ac20 0x562e69411bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57530/0 (socket says 172.21.15.112:57530) 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f877210d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939fd90 0x562e69418ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39274/0 (socket says 172.21.15.112:39274) 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f877110b640 1 -- 172.21.15.112:0/2593456676 learned_addr learned my addr 172.21.15.112:0/2593456676 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.992+0000 7f877190c640 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939a230 0x562e69411690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f877210d640 1 -- 172.21.15.112:0/2593456676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939a230 msgr2=0x562e69411690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f877210d640 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939a230 0x562e69411690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f877210d640 1 -- 172.21.15.112:0/2593456676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939ac20 msgr2=0x562e69411bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f877210d640 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939ac20 0x562e69411bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f877210d640 1 -- 172.21.15.112:0/2593456676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e69419190 con 0x562e6939fd90 2024-04-24T23:16:32.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f877110b640 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939ac20 0x562e69411bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f877190c640 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939a230 0x562e69411690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f877210d640 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939fd90 0x562e69418ba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f875c011650 tx=0x7f875c004070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1d4b59680a07ab server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f87517fa640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f87517fa640 1 -- 172.21.15.112:0/2593456676 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c022040 con 0x562e6939fd90 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f87517fa640 1 -- 172.21.15.112:0/2593456676 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f875c003970 con 0x562e6939fd90 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f8776882740 1 -- 172.21.15.112:0/2593456676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f875c011970 con 0x562e6939fd90 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f87517fa640 1 -- 172.21.15.112:0/2593456676 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f875c043050 con 0x562e6939fd90 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f8776882740 10 client.25445 fetch_fsmap learned FSMap version 28 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f8776882740 1 -- 172.21.15.112:0/2593456676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562e6941a400 con 0x562e6939fd90 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f87517fa640 1 -- 172.21.15.112:0/2593456676 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f875c011970 con 0x562e6939fd90 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.993+0000 7f87517fa640 1 -- 172.21.15.112:0/2593456676 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f875c004c00 con 0x562e6939fd90 2024-04-24T23:16:32.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 10 client.25445 fetch_fsmap finished waiting for FSMap version 28 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 10 client.25445 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 20 client.25445 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x562e6941ab50 0x562e6941cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 4 client.25445 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 -- 172.21.15.112:0/2593456676 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e69305f00 con 0x562e6941ab50 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f877190c640 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x562e6941ab50 0x562e6941cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f877190c640 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x562e6941ab50 0x562e6941cf40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf2d43a2ecb4c768 server_cookie=9670ab042416e103 in_seq=0 out_seq=0 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f87517fa640 10 client.25445 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f87517fa640 1 -- 172.21.15.112:0/2593456676 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e69305f00 con 0x562e6941ab50 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f87517fa640 10 client.25445 handle_command_reply: tid=0 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 client.25445 shutdown 2024-04-24T23:16:32.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 -- 172.21.15.112:0/2593456676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939fd90 msgr2=0x562e69418ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939fd90 0x562e69418ba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f875c011650 tx=0x7f875c004070 comp rx=0 tx=0).stop 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 -- 172.21.15.112:0/2593456676 shutdown_connections 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x562e6941ab50 0x562e6941cf40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562e6939fd90 0x562e69418ba0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f87517fa640 0 client.25445 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6939ac20 0x562e69411bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 --2- 172.21.15.112:0/2593456676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e6939a230 0x562e69411690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.994+0000 7f8776882740 1 -- 172.21.15.112:0/2593456676 >> 172.21.15.112:0/2593456676 conn(0x562e693958a0 msgr2=0x562e6939b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8776882740 1 -- 172.21.15.112:0/2593456676 shutdown_connections 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8776882740 1 -- 172.21.15.112:0/2593456676 wait complete. 2024-04-24T23:16:32.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8776882740 20 client.25445 trim_cache size 0 max 16384 2024-04-24T23:16:32.007 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:16:32.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 -- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87540a2b90 msgr2=0x7f87540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87540a2b90 0x7f87540a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f875c00fc90 tx=0x7f875c018000 comp rx=0 tx=0).stop 2024-04-24T23:16:32.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 -- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f876c101870 msgr2=0x7f876c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f876c101870 0x7f876c19c8f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f876000ea60 tx=0x7f876000ef20 comp rx=0 tx=0).stop 2024-04-24T23:16:32.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 -- 172.21.15.112:0/2054317298 shutdown_connections 2024-04-24T23:16:32.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f87540a2b90 0x7f87540a5050 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f876c102230 0x7f876c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f876c101870 0x7f876c19c8f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 --2- 172.21.15.112:0/2054317298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c100680 0x7f876c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 -- 172.21.15.112:0/2054317298 >> 172.21.15.112:0/2054317298 conn(0x7f876c0fbbf0 msgr2=0x7f876c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 -- 172.21.15.112:0/2054317298 shutdown_connections 2024-04-24T23:16:32.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:31.995+0000 7f8773b70640 1 -- 172.21.15.112:0/2054317298 wait complete. 2024-04-24T23:16:32.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:16:32.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.116+0000 7fbcd7530640 1 -- 172.21.15.112:0/3098223072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0101870 msgr2=0x7fbcd0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.116+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3098223072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0101870 0x7fbcd0101cf0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009900 tx=0x7fbccc02a410 comp rx=0 tx=0).stop 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.116+0000 7fbcd7530640 1 -- 172.21.15.112:0/3098223072 shutdown_connections 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.116+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3098223072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0102230 0x7fbcd0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.116+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3098223072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0101870 0x7fbcd0101cf0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.116+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3098223072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd0100680 0x7fbcd0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.116+0000 7fbcd7530640 1 -- 172.21.15.112:0/3098223072 >> 172.21.15.112:0/3098223072 conn(0x7fbcd00fbbf0 msgr2=0x7fbcd00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.116+0000 7fbcd7530640 1 -- 172.21.15.112:0/3098223072 shutdown_connections 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 -- 172.21.15.112:0/3098223072 wait complete. 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 Processor -- start 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 -- start start 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd0100680 0x7fbcd0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0101870 0x7fbcd0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0102230 0x7fbcd019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd010c430 con 0x7fbcd0102230 2024-04-24T23:16:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd010c2b0 con 0x7fbcd0100680 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd7530640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd010c5b0 con 0x7fbcd0101870 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd5acd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0102230 0x7fbcd019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd5acd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0102230 0x7fbcd019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57548/0 (socket says 172.21.15.112:57548) 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd5acd640 1 -- 172.21.15.112:0/3496987892 learned_addr learned my addr 172.21.15.112:0/3496987892 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.117+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd0100680 0x7fbcd0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0101870 0x7fbcd0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd5acd640 1 -- 172.21.15.112:0/3496987892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0101870 msgr2=0x7fbcd0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd5acd640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0101870 0x7fbcd0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd5acd640 1 -- 172.21.15.112:0/3496987892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd0100680 msgr2=0x7fbcd0197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd5acd640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd0100680 0x7fbcd0197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd5acd640 1 -- 172.21.15.112:0/3496987892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd0102b10 con 0x7fbcd0102230 2024-04-24T23:16:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd5acd640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0102230 0x7fbcd019f170 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800eb10 tx=0x7fbcc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcc67fc640 1 -- 172.21.15.112:0/3496987892 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8004b20 con 0x7fbcd0102230 2024-04-24T23:16:32.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd7530640 1 -- 172.21.15.112:0/3496987892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd0102e00 con 0x7fbcd0102230 2024-04-24T23:16:32.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcc67fc640 1 -- 172.21.15.112:0/3496987892 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbcc8004cc0 con 0x7fbcd0102230 2024-04-24T23:16:32.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcd7530640 1 -- 172.21.15.112:0/3496987892 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd0103370 con 0x7fbcd0102230 2024-04-24T23:16:32.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.118+0000 7fbcc67fc640 1 -- 172.21.15.112:0/3496987892 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80107f0 con 0x7fbcd0102230 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcc67fc640 1 -- 172.21.15.112:0/3496987892 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbcc8005020 con 0x7fbcd0102230 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcda23e740 1 Processor -- start 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcda23e740 1 -- start start 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 0x5633e60b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2c20 0x5633e60b7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60ba220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcda23e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633e6016670 con 0x5633e60b2c20 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcda23e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5633e5fbe970 con 0x5633e60b7d90 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcda23e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5633e6012630 con 0x5633e60b2230 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd52cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2c20 0x5633e60b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd52cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2c20 0x5633e60b7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57552/0 (socket says 172.21.15.112:57552) 2024-04-24T23:16:32.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd52cc640 1 -- 172.21.15.112:0/813264139 learned_addr learned my addr 172.21.15.112:0/813264139 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 0x5633e60b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcc67fc640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbcbc0a2b90 0x7fbcbc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd52cc640 1 -- 172.21.15.112:0/813264139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 msgr2=0x5633e60b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 0x5633e60b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd5acd640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbcbc0a2b90 0x7fbcbc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd52cc640 1 -- 172.21.15.112:0/813264139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 msgr2=0x5633e60ba220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60ba220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd52cc640 1 -- 172.21.15.112:0/813264139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e5fe2400 con 0x5633e60b2c20 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.121+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60ba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.122+0000 7fbcc67fc640 1 -- 172.21.15.112:0/3496987892 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbcc80cf950 con 0x7fbcd0102230 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.122+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2c20 0x5633e60b7720 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009ca0 tx=0x7fbcc0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424ad96176e10548 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.122+0000 7fbcbb7fe640 1 -- 172.21.15.112:0/813264139 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0004220 con 0x5633e60b2c20 2024-04-24T23:16:32.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.122+0000 7fbcd5acd640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbcbc0a2b90 0x7fbcbc0a5050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80049d0 tx=0x7fbcc806bc70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.122+0000 7fbcbb7fe640 1 -- 172.21.15.112:0/813264139 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbcc00043c0 con 0x5633e60b2c20 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.122+0000 7fbcbb7fe640 1 -- 172.21.15.112:0/813264139 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0019950 con 0x5633e60b2c20 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- 172.21.15.112:0/813264139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2c20 msgr2=0x5633e60b7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2c20 0x5633e60b7720 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009ca0 tx=0x7fbcc0011810 comp rx=0 tx=0).stop 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- 172.21.15.112:0/813264139 shutdown_connections 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60ba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2c20 0x5633e60b7720 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 --2- 172.21.15.112:0/813264139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 0x5633e60b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- 172.21.15.112:0/813264139 >> 172.21.15.112:0/813264139 conn(0x5633e60ad8a0 msgr2=0x5633e60adca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- 172.21.15.112:0/813264139 shutdown_connections 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- 172.21.15.112:0/813264139 wait complete. 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- start start 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 Processor -- start 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2c20 0x5633e60bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5633e60bad40 con 0x5633e60b2230 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5633e5fd2050 con 0x5633e60b7d90 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5633e60baec0 con 0x5633e60b2c20 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.123+0000 7fbcda23e740 10 client.0 _ll_register_callbacks cb 0x5633e5d29be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60bc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57558/0 (socket says 172.21.15.112:57558) 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 -- 172.21.15.112:0/2197122048 learned_addr learned my addr 172.21.15.112:0/2197122048 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 -- 172.21.15.112:0/2197122048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2c20 msgr2=0x5633e60bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2c20 0x5633e60bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2c20 0x5633e60bc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 -- 172.21.15.112:0/2197122048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 msgr2=0x5633e60c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 -- 172.21.15.112:0/2197122048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e5fa3dc0 con 0x5633e60b2230 2024-04-24T23:16:32.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2c20 0x5633e60bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60bc320 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbccc005ec0 tx=0x7fbccc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a191390e042f5af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/2197122048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc041070 con 0x5633e60b2230 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/2197122048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbccc005040 con 0x5633e60b2230 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcda23e740 1 -- 172.21.15.112:0/2197122048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc00043c0 con 0x5633e60b2230 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/2197122048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbccc051050 con 0x5633e60b2230 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcda23e740 10 client.16065 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.124+0000 7fbcda23e740 1 -- 172.21.15.112:0/2197122048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5633e5d2a3f0 con 0x5633e60b2230 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/2197122048 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbcc00043c0 con 0x5633e60b2230 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/2197122048 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fbccc0035f0 con 0x5633e60b2230 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcda23e740 10 client.16065 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcda23e740 10 client.16065 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcda23e740 20 client.16065 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:32.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcda23e740 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5633e60c8180 0x5633e60ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcda23e740 4 client.16065 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcda23e740 1 -- 172.21.15.112:0/2197122048 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633e6023bb0 con 0x5633e60c8180 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5633e60c8180 0x5633e60ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5633e60c8180 0x5633e60ca640 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63b8e2e76f8f7a4e server_cookie=fd8b361f86f523c in_seq=0 out_seq=0 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.125+0000 7fbcb8ff9640 10 client.16065 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/2197122048 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633e6023bb0 con 0x5633e60c8180 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcb8ff9640 10 client.16065 handle_command_reply: tid=0 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 client.16065 shutdown 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 -- 172.21.15.112:0/2197122048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 msgr2=0x5633e60bc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60bc320 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbccc005ec0 tx=0x7fbccc02a410 comp rx=0 tx=0).stop 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 -- 172.21.15.112:0/2197122048 shutdown_connections 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5633e60c8180 0x5633e60ca640 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b7d90 0x5633e60c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcb8ff9640 0 client.16065 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2c20 0x5633e60bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 --2- 172.21.15.112:0/2197122048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60bc320 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 -- 172.21.15.112:0/2197122048 >> 172.21.15.112:0/2197122048 conn(0x5633e60ad8a0 msgr2=0x5633e60acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 -- 172.21.15.112:0/2197122048 shutdown_connections 2024-04-24T23:16:32.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 1 -- 172.21.15.112:0/2197122048 wait complete. 2024-04-24T23:16:32.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.126+0000 7fbcda23e740 20 client.16065 trim_cache size 0 max 16384 2024-04-24T23:16:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcda23e740 1 Processor -- start 2024-04-24T23:16:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcda23e740 1 -- start start 2024-04-24T23:16:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 0x5633e60b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 0x5633e60c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b7d90 0x5633e60b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcda23e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01ac840 con 0x5633e60b7d90 2024-04-24T23:16:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcda23e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01ae3b0 con 0x5633e60b2c20 2024-04-24T23:16:32.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcda23e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01acf10 con 0x5633e60b2230 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b7d90 0x5633e60b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b7d90 0x5633e60b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57562/0 (socket says 172.21.15.112:57562) 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 -- 172.21.15.112:0/912246853 learned_addr learned my addr 172.21.15.112:0/912246853 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 0x5633e60c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 0x5633e60b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 -- 172.21.15.112:0/912246853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 msgr2=0x5633e60b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 0x5633e60b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 -- 172.21.15.112:0/912246853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 msgr2=0x5633e60c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 0x5633e60c3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.140+0000 7fbcd4acb640 1 -- 172.21.15.112:0/912246853 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e5df47d0 con 0x5633e60b7d90 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b7d90 0x5633e60b0cf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbccc003a10 tx=0x7fbccc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95da557e0ade95ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcbaffd640 1 -- 172.21.15.112:0/912246853 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc041040 con 0x5633e60b7d90 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcbaffd640 1 -- 172.21.15.112:0/912246853 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbccc0037f0 con 0x5633e60b7d90 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcbaffd640 1 -- 172.21.15.112:0/912246853 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc035070 con 0x5633e60b7d90 2024-04-24T23:16:32.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 -- 172.21.15.112:0/912246853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b7d90 msgr2=0x5633e60b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b7d90 0x5633e60b0cf0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbccc003a10 tx=0x7fbccc001e70 comp rx=0 tx=0).stop 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 -- 172.21.15.112:0/912246853 shutdown_connections 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b7d90 0x5633e60b0cf0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 0x5633e60c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 --2- 172.21.15.112:0/912246853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b2230 0x5633e60b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 -- 172.21.15.112:0/912246853 >> 172.21.15.112:0/912246853 conn(0x5633e60ad8a0 msgr2=0x5633e60acb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 -- 172.21.15.112:0/912246853 shutdown_connections 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.141+0000 7fbcda23e740 1 -- 172.21.15.112:0/912246853 wait complete. 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 1 -- start start 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 1 Processor -- start 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 0x5633e60c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b7d90 0x5633e60c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01adcb0 con 0x5633e60b2230 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01ad5e0 con 0x5633e60b2c20 2024-04-24T23:16:32.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd01aa820 con 0x5633e60b7d90 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcda23e740 10 client.0 _ll_register_callbacks cb 0x7fbccc005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd4acb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd4acb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57576/0 (socket says 172.21.15.112:57576) 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd4acb640 1 -- 172.21.15.112:0/3659302700 learned_addr learned my addr 172.21.15.112:0/3659302700 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 0x5633e60c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd4acb640 1 -- 172.21.15.112:0/3659302700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b7d90 msgr2=0x5633e60c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b7d90 0x5633e60c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd4acb640 1 -- 172.21.15.112:0/3659302700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 msgr2=0x5633e60c1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 0x5633e60c1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.142+0000 7fbcd4acb640 1 -- 172.21.15.112:0/3659302700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e60c3090 con 0x5633e60b2230 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcd4acb640 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60c15c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbccc003b30 tx=0x7fbccc009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f34b9bec04b3258 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:32.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/3659302700 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc041070 con 0x5633e60b2230 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/3659302700 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbccc032df0 con 0x5633e60b2230 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 1 -- 172.21.15.112:0/3659302700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbccc0037f0 con 0x5633e60b2230 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/3659302700 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbccc06a050 con 0x5633e60b2230 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 10 client.16071 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 1 -- 172.21.15.112:0/3659302700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5633e612ed40 con 0x5633e60b2230 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/3659302700 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbccc0037f0 con 0x5633e60b2230 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/3659302700 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fbccc005170 con 0x5633e60b2230 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 10 client.16071 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 20 client.16071 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5633e612f630 0x5633e6131a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 4 client.16071 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcda23e740 1 -- 172.21.15.112:0/3659302700 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633e601df00 con 0x5633e612f630 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.143+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5633e612f630 0x5633e6131a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcb8ff9640 10 client.16071 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcd52cc640 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5633e612f630 0x5633e6131a50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d450c023b1ad60ae server_cookie=be990039635a4dd1 in_seq=0 out_seq=0 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcb8ff9640 1 -- 172.21.15.112:0/3659302700 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633e601df00 con 0x5633e612f630 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcb8ff9640 10 client.16071 handle_command_reply: tid=0 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 client.16071 shutdown 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 -- 172.21.15.112:0/3659302700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 msgr2=0x5633e60c15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60c15c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbccc003b30 tx=0x7fbccc009660 comp rx=0 tx=0).stop 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 -- 172.21.15.112:0/3659302700 shutdown_connections 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x5633e612f630 0x5633e6131a50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5633e60b7d90 0x5633e60c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcb8ff9640 0 client.16071 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:16:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5633e60b2c20 0x5633e60c1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 --2- 172.21.15.112:0/3659302700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5633e60b2230 0x5633e60c15c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 -- 172.21.15.112:0/3659302700 >> 172.21.15.112:0/3659302700 conn(0x5633e60ad8a0 msgr2=0x5633e60ac2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 -- 172.21.15.112:0/3659302700 shutdown_connections 2024-04-24T23:16:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 1 -- 172.21.15.112:0/3659302700 wait complete. 2024-04-24T23:16:32.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.144+0000 7fbcda23e740 20 client.16071 trim_cache size 0 max 16384 2024-04-24T23:16:32.153 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:16:32.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 -- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbcbc0a2b90 msgr2=0x7fbcbc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbcbc0a2b90 0x7fbcbc0a5050 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbcc80049d0 tx=0x7fbcc806bc70 comp rx=0 tx=0).stop 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 -- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0102230 msgr2=0x7fbcd019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0102230 0x7fbcd019f170 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800eb10 tx=0x7fbcc800c2d0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 -- 172.21.15.112:0/3496987892 shutdown_connections 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbcbc0a2b90 0x7fbcbc0a5050 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcd0102230 0x7fbcd019f170 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0101870 0x7fbcd0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 --2- 172.21.15.112:0/3496987892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbcd0100680 0x7fbcd0197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 -- 172.21.15.112:0/3496987892 >> 172.21.15.112:0/3496987892 conn(0x7fbcd00fbbf0 msgr2=0x7fbcd00fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 -- 172.21.15.112:0/3496987892 shutdown_connections 2024-04-24T23:16:32.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.145+0000 7fbcd7530640 1 -- 172.21.15.112:0/3496987892 wait complete. 2024-04-24T23:16:32.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:16:32.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 -- 172.21.15.112:0/107980174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc101880 msgr2=0x7f9efc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 --2- 172.21.15.112:0/107980174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc101880 0x7f9efc101d00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009900 tx=0x7f9ee402a410 comp rx=0 tx=0).stop 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 -- 172.21.15.112:0/107980174 shutdown_connections 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 --2- 172.21.15.112:0/107980174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc102240 0x7f9efc108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 --2- 172.21.15.112:0/107980174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc101880 0x7f9efc101d00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 --2- 172.21.15.112:0/107980174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc100690 0x7f9efc100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 -- 172.21.15.112:0/107980174 >> 172.21.15.112:0/107980174 conn(0x7f9efc0fbc00 msgr2=0x7f9efc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 -- 172.21.15.112:0/107980174 shutdown_connections 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.260+0000 7f9f01910640 1 -- 172.21.15.112:0/107980174 wait complete. 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9f01910640 1 Processor -- start 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9f01910640 1 -- start start 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9f01910640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc100690 0x7f9efc197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9f01910640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc101880 0x7f9efc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9f01910640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc102240 0x7f9efc19f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9f01910640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10c440 con 0x7f9efc100690 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9f01910640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10c2c0 con 0x7f9efc102240 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9f01910640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc10c5c0 con 0x7f9efc101880 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc100690 0x7f9efc197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc100690 0x7f9efc197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57582/0 (socket says 172.21.15.112:57582) 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efaffd640 1 -- 172.21.15.112:0/784900670 learned_addr learned my addr 172.21.15.112:0/784900670 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc102240 0x7f9efc19f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9ef3fff640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc101880 0x7f9efc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efaffd640 1 -- 172.21.15.112:0/784900670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc101880 msgr2=0x7f9efc198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efaffd640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc101880 0x7f9efc198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efaffd640 1 -- 172.21.15.112:0/784900670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc102240 msgr2=0x7f9efc19f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efaffd640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc102240 0x7f9efc19f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.261+0000 7f9efaffd640 1 -- 172.21.15.112:0/784900670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9efc19f810 con 0x7f9efc100690 2024-04-24T23:16:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.262+0000 7f9efaffd640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc100690 0x7f9efc197c40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00d950 tx=0x7f9eec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.262+0000 7f9f0090e640 1 -- 172.21.15.112:0/784900670 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec017020 con 0x7f9efc100690 2024-04-24T23:16:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.262+0000 7f9f0090e640 1 -- 172.21.15.112:0/784900670 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9eec009c30 con 0x7f9efc100690 2024-04-24T23:16:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.262+0000 7f9f01910640 1 -- 172.21.15.112:0/784900670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9efc19fb00 con 0x7f9efc100690 2024-04-24T23:16:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.262+0000 7f9f0090e640 1 -- 172.21.15.112:0/784900670 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec00f7e0 con 0x7f9efc100690 2024-04-24T23:16:32.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.262+0000 7f9f01910640 1 -- 172.21.15.112:0/784900670 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9efc1a0020 con 0x7f9efc100690 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.264+0000 7f9f0090e640 1 -- 172.21.15.112:0/784900670 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9eec01a020 con 0x7f9efc100690 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0461e740 1 Processor -- start 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0461e740 1 -- start start 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc45a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45ac20 0x55a9fc45f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45fd90 0x55a9fc462220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0461e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9fc3be670 con 0x55a9fc45ac20 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0461e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a9fc366970 con 0x55a9fc45fd90 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0461e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a9fc3ba630 con 0x55a9fc45a230 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc45a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45ac20 0x55a9fc45f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45ac20 0x55a9fc45f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57586/0 (socket says 172.21.15.112:57586) 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9ef3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45fd90 0x55a9fc462220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9efaffd640 1 -- 172.21.15.112:0/1639137331 learned_addr learned my addr 172.21.15.112:0/1639137331 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc45a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42104/0 (socket says 172.21.15.112:42104) 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0090e640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9ee80a2b90 0x7f9ee80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9ee80a2b90 0x7f9ee80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9ef3fff640 1 -- 172.21.15.112:0/1639137331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 msgr2=0x55a9fc45a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9ef3fff640 1 --2- 172.21.15.112:0/1639137331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc45a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9ef3fff640 1 -- 172.21.15.112:0/1639137331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45ac20 msgr2=0x55a9fc45f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9ef3fff640 1 --2- 172.21.15.112:0/1639137331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45ac20 0x55a9fc45f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9ef3fff640 1 -- 172.21.15.112:0/1639137331 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fc38a400 con 0x55a9fc45fd90 2024-04-24T23:16:32.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.265+0000 7f9f0090e640 1 -- 172.21.15.112:0/784900670 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f9eec0ced70 con 0x7f9efc100690 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9ef3fff640 1 --2- 172.21.15.112:0/1639137331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45fd90 0x55a9fc462220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005ab0 tx=0x7f9ee4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25fb351e86cca093 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9ee80a2b90 0x7f9ee80a5050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55a9fc45b480 tx=0x7f9ef4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9ef1ffb640 1 -- 172.21.15.112:0/1639137331 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4040070 con 0x55a9fc45fd90 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9ef1ffb640 1 -- 172.21.15.112:0/1639137331 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ee4005020 con 0x55a9fc45fd90 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9ef1ffb640 1 -- 172.21.15.112:0/1639137331 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4035040 con 0x55a9fc45fd90 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9f0461e740 1 -- 172.21.15.112:0/1639137331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45fd90 msgr2=0x55a9fc462220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1639137331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45fd90 0x55a9fc462220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005ab0 tx=0x7f9ee4005d50 comp rx=0 tx=0).stop 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9f0461e740 1 -- 172.21.15.112:0/1639137331 shutdown_connections 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1639137331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45fd90 0x55a9fc462220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1639137331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45ac20 0x55a9fc45f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1639137331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc45a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.266+0000 7f9f0461e740 1 -- 172.21.15.112:0/1639137331 >> 172.21.15.112:0/1639137331 conn(0x55a9fc4558a0 msgr2=0x55a9fc455ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 -- 172.21.15.112:0/1639137331 shutdown_connections 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 -- 172.21.15.112:0/1639137331 wait complete. 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 -- start start 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 Processor -- start 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc464380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc4648c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a9fc462ce0 con 0x55a9fc45fd90 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a9fc37a050 con 0x55a9fc45ac20 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a9fc462e60 con 0x55a9fc45a230 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9f0461e740 10 client.0 _ll_register_callbacks cb 0x55a9fc0d1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57594/0 (socket says 172.21.15.112:57594) 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9efb7fe640 1 -- 172.21.15.112:0/1767217066 learned_addr learned my addr 172.21.15.112:0/1767217066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.267+0000 7f9ef3fff640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc4648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9efaffd640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc464380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9efb7fe640 1 -- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 msgr2=0x55a9fc464380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc464380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9efb7fe640 1 -- 172.21.15.112:0/1767217066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 msgr2=0x55a9fc4648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc4648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9efb7fe640 1 -- 172.21.15.112:0/1767217066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9fc34bdc0 con 0x55a9fc45fd90 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9efaffd640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc464380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9ef3fff640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc4648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46b890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4013450 tx=0x7f9ef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ad9233ea00af39 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9edaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9edaffd640 1 -- 172.21.15.112:0/1767217066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4025070 con 0x55a9fc45fd90 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9edaffd640 1 -- 172.21.15.112:0/1767217066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f9ef4010dc0 con 0x55a9fc45fd90 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9f0461e740 1 -- 172.21.15.112:0/1767217066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee4005020 con 0x55a9fc45fd90 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9edaffd640 1 -- 172.21.15.112:0/1767217066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ef4032050 con 0x55a9fc45fd90 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9f0461e740 10 client.16083 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9f0461e740 1 -- 172.21.15.112:0/1767217066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9fc0d23f0 con 0x55a9fc45fd90 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9edaffd640 1 -- 172.21.15.112:0/1767217066 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9ee4005020 con 0x55a9fc45fd90 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9edaffd640 1 -- 172.21.15.112:0/1767217066 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f9ef4021440 con 0x55a9fc45fd90 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.268+0000 7f9f0461e740 10 client.16083 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.269+0000 7f9f0461e740 10 client.16083 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.269+0000 7f9f0461e740 20 client.16083 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.269+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a9fc4701c0 0x55a9fc472620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.269+0000 7f9f0461e740 4 client.16083 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:32.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.269+0000 7f9f0461e740 1 -- 172.21.15.112:0/1767217066 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9fc3cbbb0 con 0x55a9fc4701c0 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.269+0000 7f9efaffd640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a9fc4701c0 0x55a9fc472620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.269+0000 7f9edaffd640 10 client.16083 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.269+0000 7f9efaffd640 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a9fc4701c0 0x55a9fc472620 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=726f62ec93d35b7d server_cookie=86e9a09c8cbad8ab in_seq=0 out_seq=0 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9edaffd640 1 -- 172.21.15.112:0/1767217066 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9fc3cbbb0 con 0x55a9fc4701c0 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9edaffd640 10 client.16083 handle_command_reply: tid=0 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 client.16083 shutdown 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 -- 172.21.15.112:0/1767217066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 msgr2=0x55a9fc46b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46b890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4013450 tx=0x7f9ef4004a40 comp rx=0 tx=0).stop 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 -- 172.21.15.112:0/1767217066 shutdown_connections 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a9fc4701c0 0x55a9fc472620 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46b890 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc4648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1767217066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45a230 0x55a9fc464380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9edaffd640 0 client.16083 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:16:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 -- 172.21.15.112:0/1767217066 >> 172.21.15.112:0/1767217066 conn(0x55a9fc4558a0 msgr2=0x55a9fc454b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 -- 172.21.15.112:0/1767217066 shutdown_connections 2024-04-24T23:16:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 1 -- 172.21.15.112:0/1767217066 wait complete. 2024-04-24T23:16:32.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.270+0000 7f9f0461e740 20 client.16083 trim_cache size 0 max 16384 2024-04-24T23:16:32.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:32 smithi112 ceph-mon[29186]: pgmap v416: 129 pgs: 129 active+clean; 100 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-04-24T23:16:32.282 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:32 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2797574602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:32.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9f0461e740 1 Processor -- start 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9f0461e740 1 -- start start 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45a230 0x55a9fc458ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc46b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45fd90 0x55a9fc459400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9f0461e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1aa6f0 con 0x55a9fc45a230 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9f0461e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1ac260 con 0x55a9fc45ac20 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9f0461e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1aadc0 con 0x55a9fc45fd90 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45a230 0x55a9fc458ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9ef3fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc46b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45a230 0x55a9fc458ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57602/0 (socket says 172.21.15.112:57602) 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9efaffd640 1 -- 172.21.15.112:0/3743157212 learned_addr learned my addr 172.21.15.112:0/3743157212 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.284+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/3743157212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45fd90 0x55a9fc459400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9efaffd640 1 -- 172.21.15.112:0/3743157212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45fd90 msgr2=0x55a9fc459400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9efaffd640 1 --2- 172.21.15.112:0/3743157212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45fd90 0x55a9fc459400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9efaffd640 1 -- 172.21.15.112:0/3743157212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 msgr2=0x55a9fc46b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9efaffd640 1 --2- 172.21.15.112:0/3743157212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc46b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9efaffd640 1 -- 172.21.15.112:0/3743157212 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9fc19c7d0 con 0x55a9fc45a230 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9efaffd640 1 --2- 172.21.15.112:0/3743157212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45a230 0x55a9fc458ec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9eec005fd0 tx=0x7f9eec0975b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fed4f290a1d3c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9ef17fa640 1 -- 172.21.15.112:0/3743157212 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0a6e60 con 0x55a9fc45a230 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9ef17fa640 1 -- 172.21.15.112:0/3743157212 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9eec009de0 con 0x55a9fc45a230 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9ef17fa640 1 -- 172.21.15.112:0/3743157212 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec097850 con 0x55a9fc45a230 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9f0461e740 1 -- 172.21.15.112:0/3743157212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45a230 msgr2=0x55a9fc458ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9f0461e740 1 --2- 172.21.15.112:0/3743157212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45a230 0x55a9fc458ec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9eec005fd0 tx=0x7f9eec0975b0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9f0461e740 1 -- 172.21.15.112:0/3743157212 shutdown_connections 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9f0461e740 1 --2- 172.21.15.112:0/3743157212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45fd90 0x55a9fc459400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9f0461e740 1 --2- 172.21.15.112:0/3743157212 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45ac20 0x55a9fc46b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.285+0000 7f9f0461e740 1 --2- 172.21.15.112:0/3743157212 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45a230 0x55a9fc458ec0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 -- 172.21.15.112:0/3743157212 >> 172.21.15.112:0/3743157212 conn(0x55a9fc4558a0 msgr2=0x55a9fc4587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 -- 172.21.15.112:0/3743157212 shutdown_connections 2024-04-24T23:16:32.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 -- 172.21.15.112:0/3743157212 wait complete. 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 -- start start 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 Processor -- start 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45a230 0x55a9fc468ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45ac20 0x55a9fc4691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1abb60 con 0x55a9fc45fd90 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1ab490 con 0x55a9fc45a230 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9efc1a86d0 con 0x55a9fc45ac20 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9f0461e740 10 client.0 _ll_register_callbacks cb 0x7f9ef4009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9efb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57604/0 (socket says 172.21.15.112:57604) 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9efb7fe640 1 -- 172.21.15.112:0/1859382538 learned_addr learned my addr 172.21.15.112:0/1859382538 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9efaffd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45a230 0x55a9fc468ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.286+0000 7f9ef3fff640 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45ac20 0x55a9fc4691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9efb7fe640 1 -- 172.21.15.112:0/1859382538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45ac20 msgr2=0x55a9fc4691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45ac20 0x55a9fc4691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9efb7fe640 1 -- 172.21.15.112:0/1859382538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45a230 msgr2=0x55a9fc468ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45a230 0x55a9fc468ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9efb7fe640 1 -- 172.21.15.112:0/1859382538 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9fc46a7e0 con 0x55a9fc45fd90 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9efaffd640 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45a230 0x55a9fc468ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9efb7fe640 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46a1f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40218e0 tx=0x7f9ef40216d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb36f5c93dd959c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9edaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9edaffd640 1 -- 172.21.15.112:0/1859382538 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef403a070 con 0x55a9fc45fd90 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9edaffd640 1 -- 172.21.15.112:0/1859382538 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f9ef4004b20 con 0x55a9fc45fd90 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 1 -- 172.21.15.112:0/1859382538 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eec009de0 con 0x55a9fc45fd90 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9edaffd640 1 -- 172.21.15.112:0/1859382538 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ef4043050 con 0x55a9fc45fd90 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 10 client.16089 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 1 -- 172.21.15.112:0/1859382538 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a9fc46bae0 con 0x55a9fc45fd90 2024-04-24T23:16:32.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9edaffd640 1 -- 172.21.15.112:0/1859382538 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9eec009de0 con 0x55a9fc45fd90 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9edaffd640 1 -- 172.21.15.112:0/1859382538 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f9ef403d020 con 0x55a9fc45fd90 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 10 client.16089 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 20 client.16089 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a9fc4d6f50 0x55a9fc4d9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 4 client.16089 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.287+0000 7f9f0461e740 1 -- 172.21.15.112:0/1859382538 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9fc3c5f00 con 0x55a9fc4d6f50 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9efaffd640 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a9fc4d6f50 0x55a9fc4d9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9efaffd640 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a9fc4d6f50 0x55a9fc4d9370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f605e35e2113f72 server_cookie=c45784a98a33346 in_seq=0 out_seq=0 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9edaffd640 10 client.16089 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9edaffd640 1 -- 172.21.15.112:0/1859382538 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9eec014020 con 0x55a9fc4d6f50 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9edaffd640 10 client.16089 handle_command_reply: tid=0 2024-04-24T23:16:32.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 client.16089 shutdown 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 -- 172.21.15.112:0/1859382538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 msgr2=0x55a9fc46a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46a1f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40218e0 tx=0x7f9ef40216d0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 -- 172.21.15.112:0/1859382538 shutdown_connections 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a9fc4d6f50 0x55a9fc4d9370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a9fc45fd90 0x55a9fc46a1f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a9fc45ac20 0x55a9fc4691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 --2- 172.21.15.112:0/1859382538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a9fc45a230 0x55a9fc468ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9edaffd640 0 client.16089 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 -- 172.21.15.112:0/1859382538 >> 172.21.15.112:0/1859382538 conn(0x55a9fc4558a0 msgr2=0x55a9fc459040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 -- 172.21.15.112:0/1859382538 shutdown_connections 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 1 -- 172.21.15.112:0/1859382538 wait complete. 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.288+0000 7f9f0461e740 20 client.16089 trim_cache size 0 max 16384 2024-04-24T23:16:32.297 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:16:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 -- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9ee80a2b90 msgr2=0x7f9ee80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9ee80a2b90 0x7f9ee80a5050 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55a9fc45b480 tx=0x7f9ef4004ab0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 -- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc100690 msgr2=0x7f9efc197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc100690 0x7f9efc197c40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00d950 tx=0x7f9eec00de10 comp rx=0 tx=0).stop 2024-04-24T23:16:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 -- 172.21.15.112:0/784900670 shutdown_connections 2024-04-24T23:16:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9ee80a2b90 0x7f9ee80a5050 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9efc102240 0x7f9efc19f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9efc101880 0x7f9efc198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 --2- 172.21.15.112:0/784900670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9efc100690 0x7f9efc197c40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 -- 172.21.15.112:0/784900670 >> 172.21.15.112:0/784900670 conn(0x7f9efc0fbc00 msgr2=0x7f9efc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 -- 172.21.15.112:0/784900670 shutdown_connections 2024-04-24T23:16:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.289+0000 7f9f01910640 1 -- 172.21.15.112:0/784900670 wait complete. 2024-04-24T23:16:32.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:16:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:32 smithi150 ceph-mon[32247]: pgmap v416: 129 pgs: 129 active+clean; 100 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-04-24T23:16:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:32 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2797574602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.403+0000 7ff0d8d42640 1 -- 172.21.15.112:0/3623672488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 msgr2=0x7ff0d4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.403+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/3623672488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 0x7ff0d4101cf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009900 tx=0x7ff0bc02a410 comp rx=0 tx=0).stop 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 -- 172.21.15.112:0/3623672488 shutdown_connections 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/3623672488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d4102230 0x7ff0d4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/3623672488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 0x7ff0d4101cf0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/3623672488 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d4100680 0x7ff0d4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 -- 172.21.15.112:0/3623672488 >> 172.21.15.112:0/3623672488 conn(0x7ff0d40fbbf0 msgr2=0x7ff0d40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 -- 172.21.15.112:0/3623672488 shutdown_connections 2024-04-24T23:16:32.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 -- 172.21.15.112:0/3623672488 wait complete. 2024-04-24T23:16:32.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 Processor -- start 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 -- start start 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d4100680 0x7ff0d4069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.404+0000 7ff0d8d42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 0x7ff0d419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d8d42640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d4102230 0x7ff0d419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d8d42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d410c480 con 0x7ff0d4101870 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d8d42640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d410c300 con 0x7ff0d4102230 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d8d42640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d410c600 con 0x7ff0d4100680 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 0x7ff0d419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 0x7ff0d419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57624/0 (socket says 172.21.15.112:57624) 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 -- 172.21.15.112:0/2405053287 learned_addr learned my addr 172.21.15.112:0/2405053287 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d4102230 0x7ff0d419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d259c640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d4100680 0x7ff0d4069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 -- 172.21.15.112:0/2405053287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d4100680 msgr2=0x7ff0d4069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d4100680 0x7ff0d4069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 -- 172.21.15.112:0/2405053287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d4102230 msgr2=0x7ff0d419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d4102230 0x7ff0d419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 -- 172.21.15.112:0/2405053287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d419f850 con 0x7ff0d4101870 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d4102230 0x7ff0d419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 0x7ff0d419c920 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0316a0 tx=0x7ff0bc032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/2405053287 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc03f030 con 0x7ff0d4101870 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/2405053287 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0bc003d00 con 0x7ff0d4101870 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d8d42640 1 -- 172.21.15.112:0/2405053287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d419fa80 con 0x7ff0d4101870 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/2405053287 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc035070 con 0x7ff0d4101870 2024-04-24T23:16:32.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.405+0000 7ff0d8d42640 1 -- 172.21.15.112:0/2405053287 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d419ff70 con 0x7ff0d4101870 2024-04-24T23:16:32.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/2405053287 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff0bc042030 con 0x7ff0d4101870 2024-04-24T23:16:32.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0dba52740 1 Processor -- start 2024-04-24T23:16:32.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0dba52740 1 -- start start 2024-04-24T23:16:32.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ec230 0x563e617ec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 0x563e617f1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617f4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0dba52740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563e61750670 con 0x563e617ec230 2024-04-24T23:16:32.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0dba52740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563e616f8970 con 0x563e617f1d90 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0dba52740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563e6174c630 con 0x563e617ecc20 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ec230 0x563e617ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ec230 0x563e617ec650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57638/0 (socket says 172.21.15.112:57638) 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0d2d9d640 1 -- 172.21.15.112:0/975214897 learned_addr learned my addr 172.21.15.112:0/975214897 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0d259c640 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 0x563e617f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0cb7fe640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0c00a2b90 0x7ff0c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0d2d9d640 1 -- 172.21.15.112:0/975214897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 msgr2=0x563e617f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 0x563e617f1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0d2d9d640 1 -- 172.21.15.112:0/975214897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 msgr2=0x563e617f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.408+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.409+0000 7ff0d2d9d640 1 -- 172.21.15.112:0/975214897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6171c400 con 0x563e617ec230 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.409+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0c00a2b90 0x7ff0c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.409+0000 7ff0cb7fe640 1 -- 172.21.15.112:0/2405053287 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff0bc0efce0 con 0x7ff0d4101870 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.409+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ec230 0x563e617ec650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0108e0 tx=0x7ff0cc010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67caa4ef32fbaccb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.409+0000 7ff0c8ff9640 1 -- 172.21.15.112:0/975214897 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc01c440 con 0x563e617ec230 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.409+0000 7ff0c8ff9640 1 -- 172.21.15.112:0/975214897 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0cc01cae0 con 0x563e617ec230 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.409+0000 7ff0c8ff9640 1 -- 172.21.15.112:0/975214897 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc011840 con 0x563e617ec230 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.409+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0c00a2b90 0x7ff0c00a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc021000 tx=0x7ff0cc01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- 172.21.15.112:0/975214897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ec230 msgr2=0x563e617ec650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ec230 0x563e617ec650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0108e0 tx=0x7ff0cc010da0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- 172.21.15.112:0/975214897 shutdown_connections 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 0x563e617f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 --2- 172.21.15.112:0/975214897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ec230 0x563e617ec650 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- 172.21.15.112:0/975214897 >> 172.21.15.112:0/975214897 conn(0x563e617e78a0 msgr2=0x563e617e7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- 172.21.15.112:0/975214897 shutdown_connections 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- 172.21.15.112:0/975214897 wait complete. 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- start start 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 Processor -- start 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 0x563e617f6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563e617f4d40 con 0x563e617ecc20 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563e6170c050 con 0x563e617f1d90 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563e617f4ec0 con 0x563e617ec230 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0dba52740 10 client.0 _ll_register_callbacks cb 0x563e61463be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.410+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57650/0 (socket says 172.21.15.112:57650) 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0cbfff640 1 -- 172.21.15.112:0/4034277023 learned_addr learned my addr 172.21.15.112:0/4034277023 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0d259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 0x563e617f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0cbfff640 1 -- 172.21.15.112:0/4034277023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 msgr2=0x563e617f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 0x563e617f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0cbfff640 1 -- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 msgr2=0x563e617fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0cbfff640 1 -- 172.21.15.112:0/4034277023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e616dddc0 con 0x563e617ecc20 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617f68d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0ed760 tx=0x7ff0bc0f33b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca59bca2697b0e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/4034277023 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc049070 con 0x563e617ecc20 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/4034277023 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff0bc0dc0e0 con 0x563e617ecc20 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0dba52740 1 -- 172.21.15.112:0/4034277023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0cc01cae0 con 0x563e617ecc20 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/4034277023 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0bc0bbc90 con 0x563e617ecc20 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0dba52740 10 client.16101 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0dba52740 1 -- 172.21.15.112:0/4034277023 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e614643f0 con 0x563e617ecc20 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.411+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/4034277023 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0cc01cae0 con 0x563e617ecc20 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/4034277023 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff0bc0b87c0 con 0x563e617ecc20 2024-04-24T23:16:32.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0dba52740 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0dba52740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0dba52740 20 client.16101 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0dba52740 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563e61802220 0x563e61804640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0dba52740 4 client.16101 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0dba52740 1 -- 172.21.15.112:0/4034277023 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e6175dbb0 con 0x563e61802220 2024-04-24T23:16:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0d259c640 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563e61802220 0x563e61804640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0d259c640 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563e61802220 0x563e61804640 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e0c1de24306d63 server_cookie=49491c07187b00cb in_seq=0 out_seq=0 2024-04-24T23:16:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.412+0000 7ff0b9ffb640 10 client.16101 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.424+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/4034277023 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e6175dbb0 con 0x563e61802220 2024-04-24T23:16:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.424+0000 7ff0b9ffb640 10 client.16101 handle_command_reply: tid=0 2024-04-24T23:16:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.424+0000 7ff0dba52740 1 client.16101 shutdown 2024-04-24T23:16:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 -- 172.21.15.112:0/4034277023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 msgr2=0x563e617f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617f68d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0ed760 tx=0x7ff0bc0f33b0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 -- 172.21.15.112:0/4034277023 shutdown_connections 2024-04-24T23:16:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563e61802220 0x563e61804640 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0b9ffb640 0 client.16101 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617f68d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 --2- 172.21.15.112:0/4034277023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 0x563e617f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 -- 172.21.15.112:0/4034277023 >> 172.21.15.112:0/4034277023 conn(0x563e617e78a0 msgr2=0x563e617e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 -- 172.21.15.112:0/4034277023 shutdown_connections 2024-04-24T23:16:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 1 -- 172.21.15.112:0/4034277023 wait complete. 2024-04-24T23:16:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.425+0000 7ff0dba52740 20 client.16101 trim_cache size 0 max 16384 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.438+0000 7ff0dba52740 1 Processor -- start 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.438+0000 7ff0dba52740 1 -- start start 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617ec230 0x563e617f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 0x563e617fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617f1d90 0x563e617ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0dba52740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41aa5e0 con 0x563e617f1d90 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0dba52740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41ac150 con 0x563e617ec230 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0dba52740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41aacb0 con 0x563e617ecc20 2024-04-24T23:16:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617f1d90 0x563e617ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617f1d90 0x563e617ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57664/0 (socket says 172.21.15.112:57664) 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 -- 172.21.15.112:0/2741743554 learned_addr learned my addr 172.21.15.112:0/2741743554 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d259c640 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617ec230 0x563e617f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 0x563e617fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 -- 172.21.15.112:0/2741743554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 msgr2=0x563e617fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 0x563e617fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 -- 172.21.15.112:0/2741743554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617ec230 msgr2=0x563e617f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617ec230 0x563e617f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 -- 172.21.15.112:0/2741743554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e6152e7d0 con 0x563e617f1d90 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617f1d90 0x563e617ea580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e720 tx=0x7ff0cc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570f598c85442803 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0bbfff640 1 -- 172.21.15.112:0/2741743554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc013050 con 0x563e617f1d90 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0bbfff640 1 -- 172.21.15.112:0/2741743554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0cc0131f0 con 0x563e617f1d90 2024-04-24T23:16:32.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.439+0000 7ff0bbfff640 1 -- 172.21.15.112:0/2741743554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc003d60 con 0x563e617f1d90 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 -- 172.21.15.112:0/2741743554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617f1d90 msgr2=0x563e617ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617f1d90 0x563e617ea580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e720 tx=0x7ff0cc00ecd0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 -- 172.21.15.112:0/2741743554 shutdown_connections 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617f1d90 0x563e617ea580 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ecc20 0x563e617fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2741743554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617ec230 0x563e617f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 -- 172.21.15.112:0/2741743554 >> 172.21.15.112:0/2741743554 conn(0x563e617e78a0 msgr2=0x563e617e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 -- 172.21.15.112:0/2741743554 shutdown_connections 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 -- 172.21.15.112:0/2741743554 wait complete. 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 -- start start 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.440+0000 7ff0dba52740 1 Processor -- start 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 0x563e617fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41aba50 con 0x563e617ecc20 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41ab380 con 0x563e617f1d90 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41a85c0 con 0x563e617ec230 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 10 client.0 _ll_register_callbacks cb 0x7ff0bc0f34b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57676/0 (socket says 172.21.15.112:57676) 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 -- 172.21.15.112:0/2019219304 learned_addr learned my addr 172.21.15.112:0/2019219304 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0d2d9d640 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0d259c640 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 0x563e617fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 -- 172.21.15.112:0/2019219304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 msgr2=0x563e617fb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 0x563e617fb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 -- 172.21.15.112:0/2019219304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 msgr2=0x563e617fc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 -- 172.21.15.112:0/2019219304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e617fcc60 con 0x563e617ecc20 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0cbfff640 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617fb660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc005ab0 tx=0x7ff0bc005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c455e554b531e1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/2019219304 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc049070 con 0x563e617ecc20 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/2019219304 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff0bc0bc3b0 con 0x563e617ecc20 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 1 -- 172.21.15.112:0/2019219304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0cc0131f0 con 0x563e617ecc20 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/2019219304 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0bc0bbab0 con 0x563e617ecc20 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 10 client.16107 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.441+0000 7ff0dba52740 1 -- 172.21.15.112:0/2019219304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e617fcc60 con 0x563e617ecc20 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/2019219304 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff0cc0131f0 con 0x563e617ecc20 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/2019219304 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff0bc0c4930 con 0x563e617ecc20 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0dba52740 10 client.16107 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0dba52740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0dba52740 20 client.16107 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:32.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563e61868fa0 0x563e6186b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0dba52740 4 client.16107 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0dba52740 1 -- 172.21.15.112:0/2019219304 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e617f6180 con 0x563e61868fa0 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0d259c640 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563e61868fa0 0x563e6186b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0b9ffb640 10 client.16107 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.442+0000 7ff0d259c640 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563e61868fa0 0x563e6186b3c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9fea8e9f41053cc server_cookie=5f9dce21e41a6481 in_seq=0 out_seq=0 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0b9ffb640 1 -- 172.21.15.112:0/2019219304 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e617f6180 con 0x563e61868fa0 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0b9ffb640 10 client.16107 handle_command_reply: tid=0 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 client.16107 shutdown 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 -- 172.21.15.112:0/2019219304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 msgr2=0x563e617fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617fb660 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc005ab0 tx=0x7ff0bc005c90 comp rx=0 tx=0).stop 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 -- 172.21.15.112:0/2019219304 shutdown_connections 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563e61868fa0 0x563e6186b3c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563e617f1d90 0x563e617fc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0b9ffb640 0 client.16107 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:32.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563e617ecc20 0x563e617fb660 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 --2- 172.21.15.112:0/2019219304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e617ec230 0x563e617fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 -- 172.21.15.112:0/2019219304 >> 172.21.15.112:0/2019219304 conn(0x563e617e78a0 msgr2=0x563e617e6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 -- 172.21.15.112:0/2019219304 shutdown_connections 2024-04-24T23:16:32.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 1 -- 172.21.15.112:0/2019219304 wait complete. 2024-04-24T23:16:32.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.443+0000 7ff0dba52740 20 client.16107 trim_cache size 0 max 16384 2024-04-24T23:16:32.452 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:16:32.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 -- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0c00a2b90 msgr2=0x7ff0c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0c00a2b90 0x7ff0c00a5050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc021000 tx=0x7ff0cc01e000 comp rx=0 tx=0).stop 2024-04-24T23:16:32.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 -- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 msgr2=0x7ff0d419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:32.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 0x7ff0d419c920 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0316a0 tx=0x7ff0bc032d70 comp rx=0 tx=0).stop 2024-04-24T23:16:32.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 -- 172.21.15.112:0/2405053287 shutdown_connections 2024-04-24T23:16:32.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff0c00a2b90 0x7ff0c00a5050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff0d4102230 0x7ff0d419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff0d4101870 0x7ff0d419c920 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 --2- 172.21.15.112:0/2405053287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0d4100680 0x7ff0d4069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 -- 172.21.15.112:0/2405053287 >> 172.21.15.112:0/2405053287 conn(0x7ff0d40fbbf0 msgr2=0x7ff0d4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 -- 172.21.15.112:0/2405053287 shutdown_connections 2024-04-24T23:16:32.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:32.444+0000 7ff0d8d42640 1 -- 172.21.15.112:0/2405053287 wait complete. 2024-04-24T23:16:32.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:32 smithi187 ceph-mon[33292]: pgmap v416: 129 pgs: 129 active+clean; 100 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 54 op/s 2024-04-24T23:16:32.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:32 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2797574602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:16:33.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:33 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:33.457 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:16:33.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:16:33.457 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:16:33.478 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:33 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:33.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:33 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 -- 172.21.15.112:0/1451943016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 msgr2=0x7fd69c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 --2- 172.21.15.112:0/1451943016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 0x7fd69c100aa0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009900 tx=0x7fd68c02a410 comp rx=0 tx=0).stop 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 -- 172.21.15.112:0/1451943016 shutdown_connections 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 --2- 172.21.15.112:0/1451943016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd69c102230 0x7fd69c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 --2- 172.21.15.112:0/1451943016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd69c101870 0x7fd69c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 --2- 172.21.15.112:0/1451943016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 0x7fd69c100aa0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 -- 172.21.15.112:0/1451943016 >> 172.21.15.112:0/1451943016 conn(0x7fd69c0fbc30 msgr2=0x7fd69c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 -- 172.21.15.112:0/1451943016 shutdown_connections 2024-04-24T23:16:33.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.562+0000 7fd6a199e640 1 -- 172.21.15.112:0/1451943016 wait complete. 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd6a199e640 1 Processor -- start 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd6a199e640 1 -- start start 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd6a199e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 0x7fd69c06a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd6a199e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd69c101870 0x7fd69c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd6a199e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd69c102230 0x7fd69c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd6a199e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c10c430 con 0x7fd69c100680 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd6a199e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c10c2b0 con 0x7fd69c102230 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd6a199e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c10c5b0 con 0x7fd69c101870 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd69affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 0x7fd69c06a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd69affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 0x7fd69c06a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57706/0 (socket says 172.21.15.112:57706) 2024-04-24T23:16:33.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd69affd640 1 -- 172.21.15.112:0/170571306 learned_addr learned my addr 172.21.15.112:0/170571306 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd69c101870 0x7fd69c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.563+0000 7fd69b7fe640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd69c102230 0x7fd69c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd69affd640 1 -- 172.21.15.112:0/170571306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd69c101870 msgr2=0x7fd69c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd69affd640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd69c101870 0x7fd69c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd69affd640 1 -- 172.21.15.112:0/170571306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd69c102230 msgr2=0x7fd69c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd69affd640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd69c102230 0x7fd69c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd69affd640 1 -- 172.21.15.112:0/170571306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c19f800 con 0x7fd69c100680 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd69affd640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 0x7fd69c06a680 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009230 tx=0x7fd68c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd6a099c640 1 -- 172.21.15.112:0/170571306 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c03f030 con 0x7fd69c100680 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd6a199e640 1 -- 172.21.15.112:0/170571306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69c19fa90 con 0x7fd69c100680 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd6a099c640 1 -- 172.21.15.112:0/170571306 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd68c003d00 con 0x7fd69c100680 2024-04-24T23:16:33.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd6a099c640 1 -- 172.21.15.112:0/170571306 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c049060 con 0x7fd69c100680 2024-04-24T23:16:33.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.564+0000 7fd6a199e640 1 -- 172.21.15.112:0/170571306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69c19ff80 con 0x7fd69c100680 2024-04-24T23:16:33.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.566+0000 7fd6a099c640 1 -- 172.21.15.112:0/170571306 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd68c042030 con 0x7fd69c100680 2024-04-24T23:16:33.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a4788740 1 Processor -- start 2024-04-24T23:16:33.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a4788740 1 -- start start 2024-04-24T23:16:33.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a099c640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6840a2b90 0x7fd6840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8786d90 0x5641f87871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f878c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f878c8f0 0x5641f878ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a4788740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641f86ea2c0 con 0x5641f878c8f0 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a4788740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5641f8692910 con 0x5641f8786d90 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a4788740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5641f86e6280 con 0x5641f8787780 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69b7fe640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6840a2b90 0x7fd6840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69affd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8786d90 0x5641f87871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f878c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd6a099c640 1 -- 172.21.15.112:0/170571306 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd68c0efca0 con 0x7fd69c100680 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f878c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42230/0 (socket says 172.21.15.112:42230) 2024-04-24T23:16:33.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69affd640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8786d90 0x5641f87871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39464/0 (socket says 172.21.15.112:39464) 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f878c8f0 0x5641f878ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69a7fc640 1 -- 172.21.15.112:0/368160378 learned_addr learned my addr 172.21.15.112:0/368160378 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69affd640 1 -- 172.21.15.112:0/368160378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 msgr2=0x5641f878c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69affd640 1 --2- 172.21.15.112:0/368160378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f878c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69affd640 1 -- 172.21.15.112:0/368160378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f878c8f0 msgr2=0x5641f878ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69affd640 1 --2- 172.21.15.112:0/368160378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f878c8f0 0x5641f878ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69affd640 1 -- 172.21.15.112:0/368160378 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f86a3fb0 con 0x5641f8786d90 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69b7fe640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6840a2b90 0x7fd6840a5050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd69400a780 tx=0x7fd69400ae00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.567+0000 7fd69affd640 1 --2- 172.21.15.112:0/368160378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8786d90 0x5641f87871b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0bbec0 tx=0x7fd68c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf0d0ab3e6420531 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd681ffb640 1 -- 172.21.15.112:0/368160378 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c035070 con 0x5641f8786d90 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd681ffb640 1 -- 172.21.15.112:0/368160378 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd68c0c5c40 con 0x5641f8786d90 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd681ffb640 1 -- 172.21.15.112:0/368160378 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0c04c0 con 0x5641f8786d90 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 -- 172.21.15.112:0/368160378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8786d90 msgr2=0x5641f87871b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:33.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 --2- 172.21.15.112:0/368160378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8786d90 0x5641f87871b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0bbec0 tx=0x7fd68c0045e0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 -- 172.21.15.112:0/368160378 shutdown_connections 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 --2- 172.21.15.112:0/368160378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f878c8f0 0x5641f878ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 --2- 172.21.15.112:0/368160378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f878c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 --2- 172.21.15.112:0/368160378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8786d90 0x5641f87871b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 -- 172.21.15.112:0/368160378 >> 172.21.15.112:0/368160378 conn(0x5641f874e950 msgr2=0x5641f8782810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 -- 172.21.15.112:0/368160378 shutdown_connections 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.568+0000 7fd6a4788740 1 -- 172.21.15.112:0/368160378 wait complete. 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 1 -- start start 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 1 Processor -- start 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f8790f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f8791480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 0x5641f87984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5641f878f660 con 0x5641f8786d90 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5641f86a5e90 con 0x5641f878c8f0 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5641f878f7e0 con 0x5641f8787780 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd6a4788740 10 client.0 _ll_register_callbacks cb 0x5641f86e1970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd69affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f8790f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd69affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f8790f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57720/0 (socket says 172.21.15.112:57720) 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd69affd640 1 -- 172.21.15.112:0/3279045574 learned_addr learned my addr 172.21.15.112:0/3279045574 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd69b7fe640 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 0x5641f87984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.569+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f8791480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd69affd640 1 -- 172.21.15.112:0/3279045574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 msgr2=0x5641f8791480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:33.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd69affd640 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f8791480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd69affd640 1 -- 172.21.15.112:0/3279045574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 msgr2=0x5641f87984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd69affd640 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 0x5641f87984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd69affd640 1 -- 172.21.15.112:0/3279045574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f84c67e0 con 0x5641f8786d90 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f8791480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd69affd640 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f8790f40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0b8b90 tx=0x7fd68c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e776253add3771f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd67affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd67affd640 1 -- 172.21.15.112:0/3279045574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c035040 con 0x5641f8786d90 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd67affd640 1 -- 172.21.15.112:0/3279045574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd68c0c0170 con 0x5641f8786d90 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 1 -- 172.21.15.112:0/3279045574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68c0c5c40 con 0x5641f8786d90 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd67affd640 1 -- 172.21.15.112:0/3279045574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd68c0bc1c0 con 0x5641f8786d90 2024-04-24T23:16:33.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 10 client.16119 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 1 -- 172.21.15.112:0/3279045574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641f8526f70 con 0x5641f8786d90 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd67affd640 1 -- 172.21.15.112:0/3279045574 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd68c0c5c40 con 0x5641f8786d90 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd67affd640 1 -- 172.21.15.112:0/3279045574 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd68c03bcb0 con 0x5641f8786d90 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 10 client.16119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 20 client.16119 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5641f879c670 0x5641f879ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 4 client.16119 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.570+0000 7fd6a4788740 1 -- 172.21.15.112:0/3279045574 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f86be5a0 con 0x5641f879c670 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.571+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5641f879c670 0x5641f879ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.571+0000 7fd67affd640 10 client.16119 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.571+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5641f879c670 0x5641f879ea60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02f74fd56989dc1 server_cookie=b73f7bd4f4ba93ba in_seq=0 out_seq=0 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd67affd640 1 -- 172.21.15.112:0/3279045574 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641f86be5a0 con 0x5641f879c670 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd67affd640 10 client.16119 handle_command_reply: tid=0 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 client.16119 shutdown 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 -- 172.21.15.112:0/3279045574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 msgr2=0x5641f8790f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f8790f40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd68c0b8b90 tx=0x7fd68c047000 comp rx=0 tx=0).stop 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 -- 172.21.15.112:0/3279045574 shutdown_connections 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5641f879c670 0x5641f879ea60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 0x5641f87984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd67affd640 0 client.16119 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8787780 0x5641f8791480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 --2- 172.21.15.112:0/3279045574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f8790f40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 -- 172.21.15.112:0/3279045574 >> 172.21.15.112:0/3279045574 conn(0x5641f874e950 msgr2=0x5641f8781950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 -- 172.21.15.112:0/3279045574 shutdown_connections 2024-04-24T23:16:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 1 -- 172.21.15.112:0/3279045574 wait complete. 2024-04-24T23:16:33.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.573+0000 7fd6a4788740 20 client.16119 trim_cache size 0 max 16384 2024-04-24T23:16:33.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.591+0000 7fd6a4788740 1 Processor -- start 2024-04-24T23:16:33.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.591+0000 7fd6a4788740 1 -- start start 2024-04-24T23:16:33.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f87911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8787780 0x5641f8798450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f878c8f0 0x5641f87a3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6a4788740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1aa5c0 con 0x5641f8786d90 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6a4788740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1ac130 con 0x5641f8787780 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6a4788740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1aac90 con 0x5641f878c8f0 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f87911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f87911c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57736/0 (socket says 172.21.15.112:57736) 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 -- 172.21.15.112:0/788564162 learned_addr learned my addr 172.21.15.112:0/788564162 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8787780 0x5641f8798450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69b7fe640 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f878c8f0 0x5641f87a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 -- 172.21.15.112:0/788564162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f878c8f0 msgr2=0x5641f87a3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f878c8f0 0x5641f87a3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 -- 172.21.15.112:0/788564162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8787780 msgr2=0x5641f8798450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:33.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8787780 0x5641f8798450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 -- 172.21.15.112:0/788564162 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f879edd0 con 0x5641f8786d90 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd69affd640 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f87911c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd68c031570 tx=0x7fd68c0dc2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8233d74aaa29ac22 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6817fa640 1 -- 172.21.15.112:0/788564162 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c035070 con 0x5641f8786d90 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6817fa640 1 -- 172.21.15.112:0/788564162 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd68c0c4d80 con 0x5641f8786d90 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.592+0000 7fd6817fa640 1 -- 172.21.15.112:0/788564162 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c03b460 con 0x5641f8786d90 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 -- 172.21.15.112:0/788564162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 msgr2=0x5641f87911c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f87911c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd68c031570 tx=0x7fd68c0dc2e0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 -- 172.21.15.112:0/788564162 shutdown_connections 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f878c8f0 0x5641f87a3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f8787780 0x5641f8798450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 --2- 172.21.15.112:0/788564162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8786d90 0x5641f87911c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 -- 172.21.15.112:0/788564162 >> 172.21.15.112:0/788564162 conn(0x5641f874e950 msgr2=0x5641f8781950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:33.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 -- 172.21.15.112:0/788564162 shutdown_connections 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 -- 172.21.15.112:0/788564162 wait complete. 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 -- start start 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.593+0000 7fd6a4788740 1 Processor -- start 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8786d90 0x5641f8797fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8787780 0x5641f87965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd6a4788740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 0x5641f8796b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd6a4788740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1aba30 con 0x5641f8787780 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd6a4788740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1ab360 con 0x5641f878c8f0 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd6a4788740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c1a85a0 con 0x5641f8786d90 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd6a4788740 10 client.0 _ll_register_callbacks cb 0x7fd694014750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8787780 0x5641f87965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8787780 0x5641f87965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57746/0 (socket says 172.21.15.112:57746) 2024-04-24T23:16:33.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 -- 172.21.15.112:0/2551759761 learned_addr learned my addr 172.21.15.112:0/2551759761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69affd640 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8786d90 0x5641f8797fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69b7fe640 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 0x5641f8796b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 -- 172.21.15.112:0/2551759761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8786d90 msgr2=0x5641f8797fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8786d90 0x5641f8797fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 -- 172.21.15.112:0/2551759761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 msgr2=0x5641f8796b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 0x5641f8796b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 -- 172.21.15.112:0/2551759761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f83be170 con 0x5641f8787780 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69affd640 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8786d90 0x5641f8797fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd69a7fc640 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8787780 0x5641f87965c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd690000c00 tx=0x7fd690004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8386c13b79d7a028 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd67affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:33.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd67affd640 1 -- 172.21.15.112:0/2551759761 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6900037f0 con 0x5641f8787780 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd67affd640 1 -- 172.21.15.112:0/2551759761 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd690003990 con 0x5641f8787780 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.594+0000 7fd6a4788740 1 -- 172.21.15.112:0/2551759761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68c0c4d80 con 0x5641f8787780 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd67affd640 1 -- 172.21.15.112:0/2551759761 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd690031050 con 0x5641f8787780 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd6a4788740 10 client.16125 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd6a4788740 1 -- 172.21.15.112:0/2551759761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641f88037c0 con 0x5641f8787780 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd67affd640 1 -- 172.21.15.112:0/2551759761 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd68c0c4d80 con 0x5641f8787780 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd67affd640 1 -- 172.21.15.112:0/2551759761 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fd690010430 con 0x5641f8787780 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd6a4788740 10 client.16125 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd6a4788740 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd6a4788740 20 client.16125 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:33.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd6a4788740 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5641f8804180 0x5641f88065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:33.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd6a4788740 4 client.16125 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:16:33.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd6a4788740 1 -- 172.21.15.112:0/2551759761 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641f86f7800 con 0x5641f8804180 2024-04-24T23:16:33.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd69affd640 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5641f8804180 0x5641f88065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:33.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd69affd640 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5641f8804180 0x5641f88065a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72b2ceeea7d0958 server_cookie=6e9b580876f585a8 in_seq=0 out_seq=0 2024-04-24T23:16:33.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:33.595+0000 7fd67affd640 10 client.16125 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:34.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.204+0000 7fd67affd640 1 -- 172.21.15.112:0/2551759761 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641f86f7800 con 0x5641f8804180 2024-04-24T23:16:34.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.204+0000 7fd67affd640 10 client.16125 handle_command_reply: tid=0 2024-04-24T23:16:34.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.204+0000 7fd6a4788740 1 client.16125 shutdown 2024-04-24T23:16:34.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 -- 172.21.15.112:0/2551759761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8787780 msgr2=0x5641f87965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8787780 0x5641f87965c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd690000c00 tx=0x7fd690004260 comp rx=0 tx=0).stop 2024-04-24T23:16:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 -- 172.21.15.112:0/2551759761 shutdown_connections 2024-04-24T23:16:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5641f8804180 0x5641f88065a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5641f878c8f0 0x5641f8796b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5641f8787780 0x5641f87965c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 --2- 172.21.15.112:0/2551759761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5641f8786d90 0x5641f8797fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd67affd640 0 client.16125 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 -- 172.21.15.112:0/2551759761 >> 172.21.15.112:0/2551759761 conn(0x5641f874e950 msgr2=0x5641f8785700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 -- 172.21.15.112:0/2551759761 shutdown_connections 2024-04-24T23:16:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 1 -- 172.21.15.112:0/2551759761 wait complete. 2024-04-24T23:16:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a4788740 20 client.16125 trim_cache size 0 max 16384 2024-04-24T23:16:34.210 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:16:34.210 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-04-24T23:16:34.210 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "fb01c782-a4c7-45fc-bacd-1be5b045613e", 2024-04-24T23:16:34.211 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-04-24T23:16:34.211 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:16:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 -- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6840a2b90 msgr2=0x7fd6840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6840a2b90 0x7fd6840a5050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd69400a780 tx=0x7fd69400ae00 comp rx=0 tx=0).stop 2024-04-24T23:16:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 -- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 msgr2=0x7fd69c06a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 0x7fd69c06a680 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009230 tx=0x7fd68c0042f0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 -- 172.21.15.112:0/170571306 shutdown_connections 2024-04-24T23:16:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd6840a2b90 0x7fd6840a5050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd69c102230 0x7fd69c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd69c101870 0x7fd69c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 --2- 172.21.15.112:0/170571306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd69c100680 0x7fd69c06a680 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.205+0000 7fd6a199e640 1 -- 172.21.15.112:0/170571306 >> 172.21.15.112:0/170571306 conn(0x7fd69c0fbc30 msgr2=0x7fd69c104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.206+0000 7fd6a199e640 1 -- 172.21.15.112:0/170571306 shutdown_connections 2024-04-24T23:16:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.206+0000 7fd6a199e640 1 -- 172.21.15.112:0/170571306 wait complete. 2024-04-24T23:16:34.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:16:34.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:16:34.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:34 smithi150 ceph-mon[32247]: pgmap v417: 129 pgs: 129 active+clean; 100 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 895 KiB/s wr, 50 op/s 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.364+0000 7feaeca88640 1 -- 172.21.15.112:0/3023898378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 msgr2=0x7feae8101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.364+0000 7feaeca88640 1 --2- 172.21.15.112:0/3023898378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 0x7feae8101ba0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7feae0009ca0 tx=0x7feae002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 -- 172.21.15.112:0/3023898378 shutdown_connections 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 --2- 172.21.15.112:0/3023898378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feae8103330 0x7feae8109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 --2- 172.21.15.112:0/3023898378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feae8102970 0x7feae8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 --2- 172.21.15.112:0/3023898378 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 0x7feae8101ba0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 -- 172.21.15.112:0/3023898378 >> 172.21.15.112:0/3023898378 conn(0x7feae80fccf0 msgr2=0x7feae80ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 -- 172.21.15.112:0/3023898378 shutdown_connections 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 -- 172.21.15.112:0/3023898378 wait complete. 2024-04-24T23:16:34.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 Processor -- start 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 -- start start 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 0x7feae80754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feae8102970 0x7feae8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feae8103330 0x7feae8074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feae810d520 con 0x7feae8101780 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feae810d3a0 con 0x7feae8102970 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.365+0000 7feaeca88640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feae810d6a0 con 0x7feae8103330 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 0x7feae80754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 0x7feae80754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57768/0 (socket says 172.21.15.112:57768) 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 -- 172.21.15.112:0/1321247615 learned_addr learned my addr 172.21.15.112:0/1321247615 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae6d9d640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feae8103330 0x7feae8074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae5d9b640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feae8102970 0x7feae8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 -- 172.21.15.112:0/1321247615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feae8103330 msgr2=0x7feae8074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feae8103330 0x7feae8074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 -- 172.21.15.112:0/1321247615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feae8102970 msgr2=0x7feae8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feae8102970 0x7feae8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 -- 172.21.15.112:0/1321247615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae8074900 con 0x7feae8101780 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feae659c640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 0x7feae80754f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feae0000f80 tx=0x7feae002f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7fead77fe640 1 -- 172.21.15.112:0/1321247615 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae002fdc0 con 0x7feae8101780 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7fead77fe640 1 -- 172.21.15.112:0/1321247615 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feae0031df0 con 0x7feae8101780 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7fead77fe640 1 -- 172.21.15.112:0/1321247615 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00469e0 con 0x7feae8101780 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feaeca88640 1 -- 172.21.15.112:0/1321247615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae81a1cf0 con 0x7feae8101780 2024-04-24T23:16:34.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.366+0000 7feaeca88640 1 -- 172.21.15.112:0/1321247615 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae81a2190 con 0x7feae8101780 2024-04-24T23:16:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.368+0000 7fead77fe640 1 -- 172.21.15.112:0/1321247615 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7feae00305c0 con 0x7feae8101780 2024-04-24T23:16:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7fead77fe640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fead00a2b90 0x7fead00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feaef793740 1 Processor -- start 2024-04-24T23:16:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feae5d9b640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fead00a2b90 0x7fead00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7fead77fe640 1 -- 172.21.15.112:0/1321247615 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7feae00f0980 con 0x7feae8101780 2024-04-24T23:16:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feaef793740 1 -- start start 2024-04-24T23:16:34.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feaef793740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2443f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feaef793740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc24444730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feaef793740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24447230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feaef793740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc243a36b0 con 0x55dc2443fc30 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feaef793740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dc2434b880 con 0x55dc24444da0 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feaef793740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dc2439f670 con 0x55dc2443f240 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.369+0000 7feae5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc24444730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2443f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24447230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2443f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:42288/0 (socket says 172.21.15.112:42288) 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae5d9b640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fead00a2b90 0x7fead00a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feae8075260 tx=0x7feadc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae659c640 1 -- 172.21.15.112:0/4074594156 learned_addr learned my addr 172.21.15.112:0/4074594156 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae5d9b640 1 -- 172.21.15.112:0/4074594156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 msgr2=0x55dc2443f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae5d9b640 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2443f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae5d9b640 1 -- 172.21.15.112:0/4074594156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 msgr2=0x55dc24447230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae5d9b640 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24447230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae5d9b640 1 -- 172.21.15.112:0/4074594156 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc241817d0 con 0x55dc2443fc30 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae659c640 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2443f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae6d9d640 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24447230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7feae5d9b640 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc24444730 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feadc005fd0 tx=0x7feadc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9cfb4423f67865 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7fead4ff9640 1 -- 172.21.15.112:0/4074594156 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc01e620 con 0x55dc2443fc30 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7fead4ff9640 1 -- 172.21.15.112:0/4074594156 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feadc01ecc0 con 0x55dc2443fc30 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.370+0000 7fead4ff9640 1 -- 172.21.15.112:0/4074594156 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc01dd80 con 0x55dc2443fc30 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- 172.21.15.112:0/4074594156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 msgr2=0x55dc24444730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc24444730 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feadc005fd0 tx=0x7feadc013000 comp rx=0 tx=0).stop 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- 172.21.15.112:0/4074594156 shutdown_connections 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24447230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc24444730 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 --2- 172.21.15.112:0/4074594156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2443f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- 172.21.15.112:0/4074594156 >> 172.21.15.112:0/4074594156 conn(0x55dc2443a8b0 msgr2=0x55dc2443acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- 172.21.15.112:0/4074594156 shutdown_connections 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- 172.21.15.112:0/4074594156 wait complete. 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- start start 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 Processor -- start 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc2443f240 0x55dc24449670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 0x55dc24449bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc24444da0 0x55dc24450bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc24447d50 con 0x55dc24444da0 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55dc2435ef50 con 0x55dc2443f240 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.371+0000 7feaef793740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dc24447ed0 con 0x55dc2443fc30 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feaef793740 10 client.0 _ll_register_callbacks cb 0x55dc2439a000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc24444da0 0x55dc24450bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc2443f240 0x55dc24449670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae5d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 0x55dc24449bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc2443f240 0x55dc24449670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:39536/0 (socket says 172.21.15.112:39536) 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc24444da0 0x55dc24450bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57788/0 (socket says 172.21.15.112:57788) 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 -- 172.21.15.112:0/3751054656 learned_addr learned my addr 172.21.15.112:0/3751054656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 -- 172.21.15.112:0/3751054656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 msgr2=0x55dc24449bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 0x55dc24449bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 -- 172.21.15.112:0/3751054656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc24444da0 msgr2=0x55dc24450bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc24444da0 0x55dc24450bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 -- 172.21.15.112:0/3751054656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc240b73f0 con 0x55dc2443f240 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae6d9d640 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc24444da0 0x55dc24450bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feae659c640 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc2443f240 0x55dc24449670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feae00ee0e0 tx=0x7feae00bd5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b818eb5df7c1921 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feacdffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feacdffb640 1 -- 172.21.15.112:0/3751054656 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00c57d0 con 0x55dc2443f240 2024-04-24T23:16:34.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feacdffb640 1 -- 172.21.15.112:0/3751054656 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7feae0046b80 con 0x55dc2443f240 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feaef793740 1 -- 172.21.15.112:0/3751054656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feadc01ecc0 con 0x55dc2443f240 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feacdffb640 1 -- 172.21.15.112:0/3751054656 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feae00c5a60 con 0x55dc2443f240 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feaef793740 10 client.25529 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.372+0000 7feaef793740 1 -- 172.21.15.112:0/3751054656 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc240770f0 con 0x55dc2443f240 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feacdffb640 1 -- 172.21.15.112:0/3751054656 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feadc01ecc0 con 0x55dc2443f240 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feacdffb640 1 -- 172.21.15.112:0/3751054656 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7feae00bc2d0 con 0x55dc2443f240 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feaef793740 10 client.25529 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feaef793740 10 client.25529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feaef793740 20 client.25529 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feaef793740 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55dc24455430 0x55dc244578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feaef793740 4 client.25529 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feaef793740 1 -- 172.21.15.112:0/3751054656 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc243b0bf0 con 0x55dc24455430 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feae5d9b640 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55dc24455430 0x55dc244578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feacdffb640 10 client.25529 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:34.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.373+0000 7feae5d9b640 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55dc24455430 0x55dc244578b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc600ce75a61a3f5 server_cookie=c4453ec9ab289c8f in_seq=0 out_seq=0 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feacdffb640 1 -- 172.21.15.112:0/3751054656 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc243b0bf0 con 0x55dc24455430 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feacdffb640 10 client.25529 handle_command_reply: tid=0 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 client.25529 shutdown 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 -- 172.21.15.112:0/3751054656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc2443f240 msgr2=0x55dc24449670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc2443f240 0x55dc24449670 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feae00ee0e0 tx=0x7feae00bd5b0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 -- 172.21.15.112:0/3751054656 shutdown_connections 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55dc24455430 0x55dc244578b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc24444da0 0x55dc24450bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feacdffb640 0 client.25529 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 0x55dc24449bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 --2- 172.21.15.112:0/3751054656 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc2443f240 0x55dc24449670 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 -- 172.21.15.112:0/3751054656 >> 172.21.15.112:0/3751054656 conn(0x55dc2443a8b0 msgr2=0x55dc24439af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 -- 172.21.15.112:0/3751054656 shutdown_connections 2024-04-24T23:16:34.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 1 -- 172.21.15.112:0/3751054656 wait complete. 2024-04-24T23:16:34.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.375+0000 7feaef793740 20 client.25529 trim_cache size 0 max 16384 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feaef793740 1 Processor -- start 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feaef793740 1 -- start start 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feaef793740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443f240 0x55dc24449b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feaef793740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 0x55dc24450b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feaef793740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24439970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feaef793740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feae81ac770 con 0x55dc2443f240 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feaef793740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feae81ae2e0 con 0x55dc24444da0 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feaef793740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feae81ace40 con 0x55dc2443fc30 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443f240 0x55dc24449b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443f240 0x55dc24449b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57804/0 (socket says 172.21.15.112:57804) 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 -- 172.21.15.112:0/3449719849 learned_addr learned my addr 172.21.15.112:0/3449719849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae5d9b640 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 0x55dc24450b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae6d9d640 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24439970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 -- 172.21.15.112:0/3449719849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 msgr2=0x55dc24450b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 0x55dc24450b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 -- 172.21.15.112:0/3449719849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 msgr2=0x55dc24439970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24439970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 -- 172.21.15.112:0/3449719849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc2443a3f0 con 0x55dc2443f240 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feae659c640 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443f240 0x55dc24449b10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feae00bcaf0 tx=0x7feae00bb300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26103c334fbaa1f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feacffff640 1 -- 172.21.15.112:0/3449719849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae00bc240 con 0x55dc2443f240 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feacffff640 1 -- 172.21.15.112:0/3449719849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feae00c0070 con 0x55dc2443f240 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.393+0000 7feacffff640 1 -- 172.21.15.112:0/3449719849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0046de0 con 0x55dc2443f240 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 -- 172.21.15.112:0/3449719849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443f240 msgr2=0x55dc24449b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443f240 0x55dc24449b10 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feae00bcaf0 tx=0x7feae00bb300 comp rx=0 tx=0).stop 2024-04-24T23:16:34.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 -- 172.21.15.112:0/3449719849 shutdown_connections 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc24439970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443fc30 0x55dc24450b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 --2- 172.21.15.112:0/3449719849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443f240 0x55dc24449b10 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 -- 172.21.15.112:0/3449719849 >> 172.21.15.112:0/3449719849 conn(0x55dc2443a8b0 msgr2=0x55dc24439260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 -- 172.21.15.112:0/3449719849 shutdown_connections 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 -- 172.21.15.112:0/3449719849 wait complete. 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 -- start start 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.394+0000 7feaef793740 1 Processor -- start 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2444e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc2444e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc2444f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feae81adbe0 con 0x55dc2443fc30 2024-04-24T23:16:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7feae81ad510 con 0x55dc24444da0 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feae81aa750 con 0x55dc2443f240 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 10 client.0 _ll_register_callbacks cb 0x7feae00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc2444e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc2444e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57820/0 (socket says 172.21.15.112:57820) 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 -- 172.21.15.112:0/4169132907 learned_addr learned my addr 172.21.15.112:0/4169132907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae659c640 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2444e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae6d9d640 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc2444f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 -- 172.21.15.112:0/4169132907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 msgr2=0x55dc2444e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2444e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 -- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 msgr2=0x55dc2444f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc2444f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 -- 172.21.15.112:0/4169132907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc2444ffe0 con 0x55dc2443fc30 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae6d9d640 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc2444f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feae5d9b640 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc2444e9e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feadc000c00 tx=0x7feadc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb385c7a95c7018 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feacdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feacdffb640 1 -- 172.21.15.112:0/4169132907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc01e970 con 0x55dc2443fc30 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feacdffb640 1 -- 172.21.15.112:0/4169132907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7feadc01eb10 con 0x55dc2443fc30 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 1 -- 172.21.15.112:0/4169132907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae00c0070 con 0x55dc2443fc30 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feacdffb640 1 -- 172.21.15.112:0/4169132907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feadc0360b0 con 0x55dc2443fc30 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 10 client.16143 fetch_fsmap learned FSMap version 29 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.395+0000 7feaef793740 1 -- 172.21.15.112:0/4169132907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc244bbd10 con 0x55dc2443fc30 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feacdffb640 1 -- 172.21.15.112:0/4169132907 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7feae00c0070 con 0x55dc2443fc30 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feacdffb640 1 -- 172.21.15.112:0/4169132907 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7feadc03e8d0 con 0x55dc2443fc30 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feaef793740 10 client.16143 fetch_fsmap finished waiting for FSMap version 29 2024-04-24T23:16:34.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feaef793740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:16:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feaef793740 20 client.16143 populate_metadata read hostname 'smithi112' 2024-04-24T23:16:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feaef793740 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55dc24455430 0x55dc244be410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feaef793740 4 client.16143 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:16:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feaef793740 1 -- 172.21.15.112:0/4169132907 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc243aaf40 con 0x55dc24455430 2024-04-24T23:16:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feae659c640 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55dc24455430 0x55dc244be410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feae659c640 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55dc24455430 0x55dc244be410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=874930730ef5308e server_cookie=1d87cde987a061f0 in_seq=0 out_seq=0 2024-04-24T23:16:34.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.396+0000 7feacdffb640 10 client.16143 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.399+0000 7feacdffb640 1 -- 172.21.15.112:0/4169132907 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55dc243aaf40 con 0x55dc24455430 2024-04-24T23:16:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.399+0000 7feacdffb640 10 client.16143 handle_command_reply: tid=0 2024-04-24T23:16:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.399+0000 7feaef793740 1 client.16143 shutdown 2024-04-24T23:16:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 -- 172.21.15.112:0/4169132907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 msgr2=0x55dc2444e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc2444e9e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7feadc000c00 tx=0x7feadc010d50 comp rx=0 tx=0).stop 2024-04-24T23:16:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 -- 172.21.15.112:0/4169132907 shutdown_connections 2024-04-24T23:16:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55dc24455430 0x55dc244be410 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55dc24444da0 0x55dc2444f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feacdffb640 0 client.16143 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc2443fc30 0x55dc2444e9e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 --2- 172.21.15.112:0/4169132907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc2443f240 0x55dc2444e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 -- 172.21.15.112:0/4169132907 >> 172.21.15.112:0/4169132907 conn(0x55dc2443a8b0 msgr2=0x55dc24449900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 -- 172.21.15.112:0/4169132907 shutdown_connections 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 1 -- 172.21.15.112:0/4169132907 wait complete. 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaef793740 20 client.16143 trim_cache size 0 max 16384 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "scrub active (189 inodes in the stack)", 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: "fb01c782-a4c7-45fc-bacd-1be5b045613e": { 2024-04-24T23:16:34.407 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-04-24T23:16:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "fb01c782-a4c7-45fc-bacd-1be5b045613e", 2024-04-24T23:16:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-04-24T23:16:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:16:34.408 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:16:34.408 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 -- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fead00a2b90 msgr2=0x7fead00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fead00a2b90 0x7fead00a5050 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7feae8075260 tx=0x7feadc00b410 comp rx=0 tx=0).stop 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 -- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 msgr2=0x7feae80754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 0x7feae80754f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feae0000f80 tx=0x7feae002f840 comp rx=0 tx=0).stop 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 -- 172.21.15.112:0/1321247615 shutdown_connections 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fead00a2b90 0x7fead00a5050 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feae8103330 0x7feae8074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feae8102970 0x7feae8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 --2- 172.21.15.112:0/1321247615 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feae8101780 0x7feae80754f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 -- 172.21.15.112:0/1321247615 >> 172.21.15.112:0/1321247615 conn(0x7feae80fccf0 msgr2=0x7feae8105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 -- 172.21.15.112:0/1321247615 shutdown_connections 2024-04-24T23:16:34.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:16:34.400+0000 7feaeca88640 1 -- 172.21.15.112:0/1321247615 wait complete. 2024-04-24T23:16:34.411 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:34 smithi112 ceph-mon[29186]: pgmap v417: 129 pgs: 129 active+clean; 100 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 895 KiB/s wr, 50 op/s 2024-04-24T23:16:34.412 INFO:tasks.cephfs.filesystem:scrub status for tag:fb01c782-a4c7-45fc-bacd-1be5b045613e - {'path': '/', 'tag': 'fb01c782-a4c7-45fc-bacd-1be5b045613e', 'options': 'recursive,force'} 2024-04-24T23:16:34.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:34 smithi187 ceph-mon[33292]: pgmap v417: 129 pgs: 129 active+clean; 100 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 895 KiB/s wr, 50 op/s 2024-04-24T23:16:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:35 smithi150 ceph-mon[32247]: scrub queued for path: / 2024-04-24T23:16:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:35 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:35 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:16:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:35 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:16:35.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:35 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:35.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:35 smithi187 ceph-mon[33292]: scrub queued for path: / 2024-04-24T23:16:35.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:35 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:35.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:35 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:16:35.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:35 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:16:35.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:35 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:35 smithi112 ceph-mon[29186]: scrub queued for path: / 2024-04-24T23:16:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:35 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:35 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:16:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:35 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:16:35.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:35 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:36 smithi150 ceph-mon[32247]: pgmap v418: 129 pgs: 129 active+clean; 107 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-04-24T23:16:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:36 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:36.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:36 smithi187 ceph-mon[33292]: pgmap v418: 129 pgs: 129 active+clean; 107 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-04-24T23:16:36.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:36 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:36.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:36 smithi112 ceph-mon[29186]: pgmap v418: 129 pgs: 129 active+clean; 107 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-04-24T23:16:36.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:36 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:16:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:38 smithi150 ceph-mon[32247]: pgmap v419: 129 pgs: 129 active+clean; 113 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 57 op/s 2024-04-24T23:16:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:38 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:38.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:38 smithi187 ceph-mon[33292]: pgmap v419: 129 pgs: 129 active+clean; 113 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 57 op/s 2024-04-24T23:16:38.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:38 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:38 smithi112 ceph-mon[29186]: pgmap v419: 129 pgs: 129 active+clean; 113 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 57 op/s 2024-04-24T23:16:38.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:38 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:39.037 INFO:tasks.workunit.client.0.smithi112.stderr:+ sync 2024-04-24T23:16:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:40 smithi150 ceph-mon[32247]: pgmap v420: 129 pgs: 129 active+clean; 113 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 33 op/s 2024-04-24T23:16:40.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:40 smithi187 ceph-mon[33292]: pgmap v420: 129 pgs: 129 active+clean; 113 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 33 op/s 2024-04-24T23:16:40.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:40 smithi112 ceph-mon[29186]: pgmap v420: 129 pgs: 129 active+clean; 113 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 33 op/s 2024-04-24T23:16:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:42 smithi150 ceph-mon[32247]: pgmap v421: 129 pgs: 129 active+clean; 155 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 394 op/s 2024-04-24T23:16:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:42 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:16:42.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:42 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:16:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:42 smithi187 ceph-mon[33292]: pgmap v421: 129 pgs: 129 active+clean; 155 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 394 op/s 2024-04-24T23:16:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:42 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:16:42.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:42 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:16:42.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:42 smithi112 ceph-mon[29186]: pgmap v421: 129 pgs: 129 active+clean; 155 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 394 op/s 2024-04-24T23:16:42.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:42 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:16:42.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:42 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:16:43.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:43 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:43 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:43.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:43 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:44 smithi187 ceph-mon[33292]: pgmap v422: 129 pgs: 129 active+clean; 163 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 467 op/s 2024-04-24T23:16:44.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:44 smithi112 ceph-mon[29186]: pgmap v422: 129 pgs: 129 active+clean; 163 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 467 op/s 2024-04-24T23:16:44.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:44 smithi150 ceph-mon[32247]: pgmap v422: 129 pgs: 129 active+clean; 163 MiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.9 MiB/s wr, 467 op/s 2024-04-24T23:16:46.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:46 smithi187 ceph-mon[33292]: pgmap v423: 129 pgs: 129 active+clean; 175 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.3 MiB/s wr, 699 op/s 2024-04-24T23:16:46.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:46 smithi112 ceph-mon[29186]: pgmap v423: 129 pgs: 129 active+clean; 175 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.3 MiB/s wr, 699 op/s 2024-04-24T23:16:46.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:46 smithi150 ceph-mon[32247]: pgmap v423: 129 pgs: 129 active+clean; 175 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.3 MiB/s wr, 699 op/s 2024-04-24T23:16:47.175 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar c linux-4.0.5 2024-04-24T23:16:47.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:47 smithi187 ceph-mon[33292]: pgmap v424: 129 pgs: 129 active+clean; 176 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 913 op/s 2024-04-24T23:16:47.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:47 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:47.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:47 smithi112 ceph-mon[29186]: pgmap v424: 129 pgs: 129 active+clean; 176 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 913 op/s 2024-04-24T23:16:47.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:47 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:47.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:47 smithi150 ceph-mon[32247]: pgmap v424: 129 pgs: 129 active+clean; 176 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.7 MiB/s wr, 913 op/s 2024-04-24T23:16:47.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:47 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:47.607 INFO:tasks.workunit.client.0.smithi112.stderr:+ rm -rf linux-4.0.5 2024-04-24T23:16:50.232 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:49 smithi112 ceph-mon[29186]: pgmap v425: 129 pgs: 129 active+clean; 175 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 925 op/s 2024-04-24T23:16:50.235 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:49 smithi187 ceph-mon[33292]: pgmap v425: 129 pgs: 129 active+clean; 175 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 925 op/s 2024-04-24T23:16:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:49 smithi150 ceph-mon[32247]: pgmap v425: 129 pgs: 129 active+clean; 175 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 925 op/s 2024-04-24T23:16:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:51 smithi187 ceph-mon[33292]: pgmap v426: 129 pgs: 129 active+clean; 181 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.6 MiB/s wr, 953 op/s 2024-04-24T23:16:52.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:51 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:51 smithi112 ceph-mon[29186]: pgmap v426: 129 pgs: 129 active+clean; 181 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.6 MiB/s wr, 953 op/s 2024-04-24T23:16:52.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:51 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:51 smithi150 ceph-mon[32247]: pgmap v426: 129 pgs: 129 active+clean; 181 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.6 MiB/s wr, 953 op/s 2024-04-24T23:16:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:51 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:54.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:53 smithi187 ceph-mon[33292]: pgmap v427: 129 pgs: 129 active+clean; 180 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 595 op/s 2024-04-24T23:16:54.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:53 smithi112 ceph-mon[29186]: pgmap v427: 129 pgs: 129 active+clean; 180 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 595 op/s 2024-04-24T23:16:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:53 smithi150 ceph-mon[32247]: pgmap v427: 129 pgs: 129 active+clean; 180 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 595 op/s 2024-04-24T23:16:56.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:55 smithi187 ceph-mon[33292]: pgmap v428: 129 pgs: 129 active+clean; 180 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 559 op/s 2024-04-24T23:16:56.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:55 smithi112 ceph-mon[29186]: pgmap v428: 129 pgs: 129 active+clean; 180 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 559 op/s 2024-04-24T23:16:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:55 smithi150 ceph-mon[32247]: pgmap v428: 129 pgs: 129 active+clean; 180 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 559 op/s 2024-04-24T23:16:58.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:57 smithi187 ceph-mon[33292]: pgmap v429: 129 pgs: 129 active+clean; 183 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 452 op/s 2024-04-24T23:16:58.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:57 smithi112 ceph-mon[29186]: pgmap v429: 129 pgs: 129 active+clean; 183 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 452 op/s 2024-04-24T23:16:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:57 smithi150 ceph-mon[32247]: pgmap v429: 129 pgs: 129 active+clean; 183 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 452 op/s 2024-04-24T23:16:59.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:58 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:59.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:58 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:59.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:58 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:16:59.741 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar xf doc.tar 2024-04-24T23:17:00.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:16:59 smithi187 ceph-mon[33292]: pgmap v430: 129 pgs: 129 active+clean; 181 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-04-24T23:17:00.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:16:59 smithi112 ceph-mon[29186]: pgmap v430: 129 pgs: 129 active+clean; 181 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-04-24T23:17:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:16:59 smithi150 ceph-mon[32247]: pgmap v430: 129 pgs: 129 active+clean; 181 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-04-24T23:17:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:01 smithi187 ceph-mon[33292]: pgmap v431: 129 pgs: 129 active+clean; 180 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 353 op/s 2024-04-24T23:17:02.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:01 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:02.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:01 smithi112 ceph-mon[29186]: pgmap v431: 129 pgs: 129 active+clean; 180 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 353 op/s 2024-04-24T23:17:02.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:01 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:01 smithi150 ceph-mon[32247]: pgmap v431: 129 pgs: 129 active+clean; 180 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 353 op/s 2024-04-24T23:17:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:01 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:04.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:03 smithi187 ceph-mon[33292]: pgmap v432: 129 pgs: 129 active+clean; 179 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 327 op/s 2024-04-24T23:17:04.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:03 smithi112 ceph-mon[29186]: pgmap v432: 129 pgs: 129 active+clean; 179 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 327 op/s 2024-04-24T23:17:04.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:03 smithi150 ceph-mon[32247]: pgmap v432: 129 pgs: 129 active+clean; 179 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 327 op/s 2024-04-24T23:17:04.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:17:04.414 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.512+0000 7f2dcef96640 1 -- 172.21.15.112:0/3251947924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc81074e0 msgr2=0x7f2dc81098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.512+0000 7f2dcef96640 1 --2- 172.21.15.112:0/3251947924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc81074e0 0x7f2dc81098d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009ca0 tx=0x7f2dc002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 -- 172.21.15.112:0/3251947924 shutdown_connections 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 --2- 172.21.15.112:0/3251947924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc81074e0 0x7f2dc81098d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 --2- 172.21.15.112:0/3251947924 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc8104bb0 0x7f2dc8106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 --2- 172.21.15.112:0/3251947924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc8102280 0x7f2dc8104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 -- 172.21.15.112:0/3251947924 >> 172.21.15.112:0/3251947924 conn(0x7f2dc80fbbf0 msgr2=0x7f2dc80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 -- 172.21.15.112:0/3251947924 shutdown_connections 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 -- 172.21.15.112:0/3251947924 wait complete. 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 Processor -- start 2024-04-24T23:17:04.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 -- start start 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc8102280 0x7f2dc8101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc8104bb0 0x7f2dc8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc81074e0 0x7f2dc8100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc810d550 con 0x7f2dc8104bb0 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc810d3d0 con 0x7f2dc8102280 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dcef96640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc810d6d0 con 0x7f2dc81074e0 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc8104bb0 0x7f2dc8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc8104bb0 0x7f2dc8100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48246/0 (socket says 172.21.15.112:48246) 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.513+0000 7f2dc7fff640 1 -- 172.21.15.112:0/1258966964 learned_addr learned my addr 172.21.15.112:0/1258966964 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dcd533640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc81074e0 0x7f2dc8100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dccd32640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc8102280 0x7f2dc8101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc7fff640 1 -- 172.21.15.112:0/1258966964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc81074e0 msgr2=0x7f2dc8100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc81074e0 0x7f2dc8100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc7fff640 1 -- 172.21.15.112:0/1258966964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc8102280 msgr2=0x7f2dc8101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc8102280 0x7f2dc8101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc7fff640 1 -- 172.21.15.112:0/1258966964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc8101030 con 0x7f2dc8104bb0 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dcd533640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc81074e0 0x7f2dc8100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc8104bb0 0x7f2dc8100230 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2db000d770 tx=0x7f2db000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc5ffb640 1 -- 172.21.15.112:0/1258966964 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0017070 con 0x7f2dc8104bb0 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc5ffb640 1 -- 172.21.15.112:0/1258966964 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2db0009c30 con 0x7f2dc8104bb0 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dcef96640 1 -- 172.21.15.112:0/1258966964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc81a1cc0 con 0x7f2dc8104bb0 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dc5ffb640 1 -- 172.21.15.112:0/1258966964 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db000f6b0 con 0x7f2dc8104bb0 2024-04-24T23:17:04.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.514+0000 7f2dcef96640 1 -- 172.21.15.112:0/1258966964 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc81a2190 con 0x7f2dc8104bb0 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.516+0000 7f2dc5ffb640 1 -- 172.21.15.112:0/1258966964 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2db001a070 con 0x7f2dc8104bb0 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dd1d76740 1 Processor -- start 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dd1d76740 1 -- start start 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc5ffb640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2db40a2b90 0x7f2db40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d10d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d112730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d112da0 0x56122d115230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56122d0716b0 con 0x56122d10dc30 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56122d019880 con 0x56122d112da0 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56122d06d670 con 0x56122d10d240 2024-04-24T23:17:04.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc5ffb640 1 -- 172.21.15.112:0/1258966964 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f2db00ceff0 con 0x7f2dc8104bb0 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dcd533640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2db40a2b90 0x7f2db40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48258/0 (socket says 172.21.15.112:48258) 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 -- 172.21.15.112:0/2528926192 learned_addr learned my addr 172.21.15.112:0/2528926192 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dccd32640 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d10d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dcd533640 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d112da0 0x56122d115230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 -- 172.21.15.112:0/2528926192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 msgr2=0x56122d10d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d10d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 -- 172.21.15.112:0/2528926192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d112da0 msgr2=0x56122d115230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d112da0 0x56122d115230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 -- 172.21.15.112:0/2528926192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122ce4f7d0 con 0x56122d10dc30 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dcd533640 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d112da0 0x56122d115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d112730 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2db0095fd0 tx=0x7f2db009b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cdfac4bede0776 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.517+0000 7f2dcd533640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2db40a2b90 0x7f2db40a5050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005fe0 tx=0x7f2dc0005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dbeffd640 1 -- 172.21.15.112:0/2528926192 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0021070 con 0x56122d10dc30 2024-04-24T23:17:04.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dbeffd640 1 -- 172.21.15.112:0/2528926192 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2db009f810 con 0x56122d10dc30 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dbeffd640 1 -- 172.21.15.112:0/2528926192 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db009a8b0 con 0x56122d10dc30 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2528926192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 msgr2=0x56122d112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d112730 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2db0095fd0 tx=0x7f2db009b360 comp rx=0 tx=0).stop 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2528926192 shutdown_connections 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d112da0 0x56122d115230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d112730 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2528926192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2528926192 >> 172.21.15.112:0/2528926192 conn(0x56122d1088b0 msgr2=0x56122d108cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2528926192 shutdown_connections 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.518+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2528926192 wait complete. 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 1 -- start start 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 1 Processor -- start 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d1175d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d117b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56122d115c90 con 0x56122d112da0 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56122d02cf50 con 0x56122d10dc30 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56122d115e10 con 0x56122d10d240 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dd1d76740 10 client.0 _ll_register_callbacks cb 0x56122d068000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dcd533640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dcd533640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48268/0 (socket says 172.21.15.112:48268) 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dcd533640 1 -- 172.21.15.112:0/1770345163 learned_addr learned my addr 172.21.15.112:0/1770345163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dccd32640 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d1175d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d117b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dcd533640 1 -- 172.21.15.112:0/1770345163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 msgr2=0x56122d1175d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dcd533640 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d1175d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dcd533640 1 -- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 msgr2=0x56122d117b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dcd533640 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d117b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dcd533640 1 -- 172.21.15.112:0/1770345163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122cd853f0 con 0x56122d112da0 2024-04-24T23:17:04.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.519+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d117b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dcd533640 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11eb50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0035000 tx=0x7f2dc00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c8ae92f158ffb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2da7fff640 1 -- 172.21.15.112:0/1770345163 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0003e00 con 0x56122d112da0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2da7fff640 1 -- 172.21.15.112:0/1770345163 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f2dc00045d0 con 0x56122d112da0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 1 -- 172.21.15.112:0/1770345163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db009f810 con 0x56122d112da0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2da7fff640 1 -- 172.21.15.112:0/1770345163 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2dc0058050 con 0x56122d112da0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 10 client.16155 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 1 -- 172.21.15.112:0/1770345163 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56122cd450f0 con 0x56122d112da0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2da7fff640 1 -- 172.21.15.112:0/1770345163 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2db009f810 con 0x56122d112da0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2da7fff640 1 -- 172.21.15.112:0/1770345163 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f2dc00305c0 con 0x56122d112da0 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 10 client.16155 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 20 client.16155 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:04.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56122d123500 0x56122d125980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 4 client.16155 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.520+0000 7f2dd1d76740 1 -- 172.21.15.112:0/1770345163 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56122d07ebf0 con 0x56122d123500 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.521+0000 7f2dccd32640 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56122d123500 0x56122d125980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.521+0000 7f2da7fff640 10 client.16155 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.521+0000 7f2dccd32640 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56122d123500 0x56122d125980 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cfc7970d320af85 server_cookie=4f0cfdcfffa425cf in_seq=0 out_seq=0 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2da7fff640 1 -- 172.21.15.112:0/1770345163 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56122d07ebf0 con 0x56122d123500 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2da7fff640 10 client.16155 handle_command_reply: tid=0 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 client.16155 shutdown 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 -- 172.21.15.112:0/1770345163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 msgr2=0x56122d11eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11eb50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0035000 tx=0x7f2dc00095c0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 -- 172.21.15.112:0/1770345163 shutdown_connections 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56122d123500 0x56122d125980 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11eb50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2da7fff640 0 client.16155 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d117b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/1770345163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d1175d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 -- 172.21.15.112:0/1770345163 >> 172.21.15.112:0/1770345163 conn(0x56122d1088b0 msgr2=0x56122d107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 -- 172.21.15.112:0/1770345163 shutdown_connections 2024-04-24T23:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 1 -- 172.21.15.112:0/1770345163 wait complete. 2024-04-24T23:17:04.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.522+0000 7f2dd1d76740 20 client.16155 trim_cache size 0 max 16384 2024-04-24T23:17:04.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dd1d76740 1 Processor -- start 2024-04-24T23:17:04.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dd1d76740 1 -- start start 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10d240 0x56122d116bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d11eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d112da0 0x56122d10b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81ac860 con 0x56122d10dc30 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81ae4c0 con 0x56122d10d240 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81acf30 con 0x56122d112da0 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d11eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dc7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d11eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48276/0 (socket says 172.21.15.112:48276) 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.539+0000 7f2dc7fff640 1 -- 172.21.15.112:0/3761106481 learned_addr learned my addr 172.21.15.112:0/3761106481 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dccd32640 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10d240 0x56122d116bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dc7fff640 1 -- 172.21.15.112:0/3761106481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d112da0 msgr2=0x56122d10b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dcd533640 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d112da0 0x56122d10b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d112da0 0x56122d10b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dc7fff640 1 -- 172.21.15.112:0/3761106481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10d240 msgr2=0x56122d116bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10d240 0x56122d116bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dc7fff640 1 -- 172.21.15.112:0/3761106481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122d1083f0 con 0x56122d10dc30 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d11eb10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2db00978c0 tx=0x7f2db009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c84dee23cd490 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dbe7fc640 1 -- 172.21.15.112:0/3761106481 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db00b3070 con 0x56122d10dc30 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dbe7fc640 1 -- 172.21.15.112:0/3761106481 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2db009f810 con 0x56122d10dc30 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dbe7fc640 1 -- 172.21.15.112:0/3761106481 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0021040 con 0x56122d10dc30 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dd1d76740 1 -- 172.21.15.112:0/3761106481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 msgr2=0x56122d11eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.540+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d11eb10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2db00978c0 tx=0x7f2db009ae70 comp rx=0 tx=0).stop 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 -- 172.21.15.112:0/3761106481 shutdown_connections 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d112da0 0x56122d10b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d10dc30 0x56122d11eb10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/3761106481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10d240 0x56122d116bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 -- 172.21.15.112:0/3761106481 >> 172.21.15.112:0/3761106481 conn(0x56122d1088b0 msgr2=0x56122d10f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 -- 172.21.15.112:0/3761106481 shutdown_connections 2024-04-24T23:17:04.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 -- 172.21.15.112:0/3761106481 wait complete. 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 -- start start 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 Processor -- start 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d11c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d11c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81add30 con 0x56122d112da0 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81ad630 con 0x56122d10dc30 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81aa810 con 0x56122d10d240 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dd1d76740 10 client.0 _ll_register_callbacks cb 0x7f2dc0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dcd533640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dcd533640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48290/0 (socket says 172.21.15.112:48290) 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dcd533640 1 -- 172.21.15.112:0/2354920620 learned_addr learned my addr 172.21.15.112:0/2354920620 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dccd32640 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d11c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.541+0000 7f2dc7fff640 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d11c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dcd533640 1 -- 172.21.15.112:0/2354920620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 msgr2=0x56122d11c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:04.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dcd533640 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d11c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dcd533640 1 -- 172.21.15.112:0/2354920620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 msgr2=0x56122d11c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dcd533640 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d11c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dcd533640 1 -- 172.21.15.112:0/2354920620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122d11dd00 con 0x56122d112da0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dcd533640 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11d710 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0030a80 tx=0x7f2dc0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca904af6642e0ef server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2da7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2da7fff640 1 -- 172.21.15.112:0/2354920620 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0062020 con 0x56122d112da0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2da7fff640 1 -- 172.21.15.112:0/2354920620 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f2dc0030c70 con 0x56122d112da0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2354920620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db009f810 con 0x56122d112da0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2da7fff640 1 -- 172.21.15.112:0/2354920620 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2dc00307d0 con 0x56122d112da0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 10 client.16161 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2354920620 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56122d189ac0 con 0x56122d112da0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2da7fff640 1 -- 172.21.15.112:0/2354920620 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f2db009f810 con 0x56122d112da0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2da7fff640 1 -- 172.21.15.112:0/2354920620 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f2dc002f7b0 con 0x56122d112da0 2024-04-24T23:17:04.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 10 client.16161 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 20 client.16161 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56122d123500 0x56122d18c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 4 client.16161 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.542+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2354920620 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56122d078f40 con 0x56122d123500 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dccd32640 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56122d123500 0x56122d18c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2da7fff640 10 client.16161 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dccd32640 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56122d123500 0x56122d18c5e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b073af923690d4 server_cookie=35599dc847a5ef7b in_seq=0 out_seq=0 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2da7fff640 1 -- 172.21.15.112:0/2354920620 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56122d078f40 con 0x56122d123500 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2da7fff640 10 client.16161 handle_command_reply: tid=0 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 client.16161 shutdown 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2354920620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 msgr2=0x56122d11d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11d710 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0030a80 tx=0x7f2dc0005ce0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2354920620 shutdown_connections 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x56122d123500 0x56122d18c5e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56122d112da0 0x56122d11d710 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56122d10dc30 0x56122d11c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2da7fff640 0 client.16161 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 --2- 172.21.15.112:0/2354920620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56122d10d240 0x56122d11c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2354920620 >> 172.21.15.112:0/2354920620 conn(0x56122d1088b0 msgr2=0x56122d107260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2354920620 shutdown_connections 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.543+0000 7f2dd1d76740 1 -- 172.21.15.112:0/2354920620 wait complete. 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dd1d76740 20 client.16161 trim_cache size 0 max 16384 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-04-24T23:17:04.552 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 -- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2db40a2b90 msgr2=0x7f2db40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2db40a2b90 0x7f2db40a5050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005fe0 tx=0x7f2dc0005ef0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 -- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc8104bb0 msgr2=0x7f2dc8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc8104bb0 0x7f2dc8100230 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2db000d770 tx=0x7f2db000dc30 comp rx=0 tx=0).stop 2024-04-24T23:17:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 -- 172.21.15.112:0/1258966964 shutdown_connections 2024-04-24T23:17:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f2db40a2b90 0x7f2db40a5050 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2dc81074e0 0x7f2dc8100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dc8104bb0 0x7f2dc8100230 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 --2- 172.21.15.112:0/1258966964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc8102280 0x7f2dc8101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 -- 172.21.15.112:0/1258966964 >> 172.21.15.112:0/1258966964 conn(0x7f2dc80fbbf0 msgr2=0x7f2dc8102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 -- 172.21.15.112:0/1258966964 shutdown_connections 2024-04-24T23:17:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.544+0000 7f2dcef96640 1 -- 172.21.15.112:0/1258966964 wait complete. 2024-04-24T23:17:04.556 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:17:04.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:17:04.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 -- 172.21.15.112:0/615107791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8101870 msgr2=0x7fb6c8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 --2- 172.21.15.112:0/615107791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8101870 0x7fb6c8101cf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009900 tx=0x7fb6b002a410 comp rx=0 tx=0).stop 2024-04-24T23:17:04.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 -- 172.21.15.112:0/615107791 shutdown_connections 2024-04-24T23:17:04.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 --2- 172.21.15.112:0/615107791 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c8102230 0x7fb6c8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 --2- 172.21.15.112:0/615107791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8101870 0x7fb6c8101cf0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 --2- 172.21.15.112:0/615107791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6c8100680 0x7fb6c8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 -- 172.21.15.112:0/615107791 >> 172.21.15.112:0/615107791 conn(0x7fb6c80fbbf0 msgr2=0x7fb6c80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 -- 172.21.15.112:0/615107791 shutdown_connections 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 -- 172.21.15.112:0/615107791 wait complete. 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 Processor -- start 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 -- start start 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8100680 0x7fb6c8069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.652+0000 7fb6cde31640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c8101870 0x7fb6c819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6cde31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6c8102230 0x7fb6c819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6cde31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810c430 con 0x7fb6c8100680 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6cde31640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810c2b0 con 0x7fb6c8101870 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6cde31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c810c5b0 con 0x7fb6c8102230 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8100680 0x7fb6c8069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8100680 0x7fb6c8069440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48304/0 (socket says 172.21.15.112:48304) 2024-04-24T23:17:04.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 -- 172.21.15.112:0/3699925912 learned_addr learned my addr 172.21.15.112:0/3699925912 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6bffff640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c8101870 0x7fb6c819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 -- 172.21.15.112:0/3699925912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6c8102230 msgr2=0x7fb6c819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6c8102230 0x7fb6c819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 -- 172.21.15.112:0/3699925912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c8101870 msgr2=0x7fb6c819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c8101870 0x7fb6c819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 -- 172.21.15.112:0/3699925912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c819f740 con 0x7fb6c8100680 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c77fe640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8100680 0x7fb6c8069440 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c970 tx=0x7fb6b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c57fa640 1 -- 172.21.15.112:0/3699925912 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8007c10 con 0x7fb6c8100680 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c57fa640 1 -- 172.21.15.112:0/3699925912 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb6b8007db0 con 0x7fb6c8100680 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c819f9d0 con 0x7fb6c8100680 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6c57fa640 1 -- 172.21.15.112:0/3699925912 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b800fa40 con 0x7fb6c8100680 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.653+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c819fef0 con 0x7fb6c8100680 2024-04-24T23:17:04.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.654+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c81078d0 con 0x7fb6c8100680 2024-04-24T23:17:04.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.656+0000 7fb6c57fa640 1 -- 172.21.15.112:0/3699925912 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb6b800fbe0 con 0x7fb6c8100680 2024-04-24T23:17:04.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.656+0000 7fb6c57fa640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb6b40a2b90 0x7fb6b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.657+0000 7fb6bffff640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb6b40a2b90 0x7fb6b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.657+0000 7fb6c57fa640 1 -- 172.21.15.112:0/3699925912 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb6b80cfb10 con 0x7fb6c8100680 2024-04-24T23:17:04.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.657+0000 7fb6bffff640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb6b40a2b90 0x7fb6b40a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0000c00 tx=0x7fb6b0031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.660+0000 7fb6c57fa640 1 -- 172.21.15.112:0/3699925912 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb6b800e3b0 con 0x7fb6c8100680 2024-04-24T23:17:04.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.874+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6c802d390 con 0x7fb6c8100680 2024-04-24T23:17:04.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.875+0000 7fb6c57fa640 1 -- 172.21.15.112:0/3699925912 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11958 (secure 0 0 0) 0x7fb6b809c7a0 con 0x7fb6c8100680 2024-04-24T23:17:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:17:04.877 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:16:59.958458+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24482,24490,24425,24424]},"id":1}]} 2024-04-24T23:17:04.877 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 35 2024-04-24T23:17:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb6b40a2b90 msgr2=0x7fb6b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb6b40a2b90 0x7fb6b40a5050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0000c00 tx=0x7fb6b0031640 comp rx=0 tx=0).stop 2024-04-24T23:17:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8100680 msgr2=0x7fb6c8069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8100680 0x7fb6c8069440 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c970 tx=0x7fb6b800ce30 comp rx=0 tx=0).stop 2024-04-24T23:17:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 shutdown_connections 2024-04-24T23:17:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb6b40a2b90 0x7fb6b40a5050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6c8102230 0x7fb6c819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb6c8101870 0x7fb6c819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 --2- 172.21.15.112:0/3699925912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c8100680 0x7fb6c8069440 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 >> 172.21.15.112:0/3699925912 conn(0x7fb6c80fbbf0 msgr2=0x7fb6c8104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 shutdown_connections 2024-04-24T23:17:04.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.877+0000 7fb6cde31640 1 -- 172.21.15.112:0/3699925912 wait complete. 2024-04-24T23:17:04.889 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 -- 172.21.15.112:0/4051036384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a81006a0 msgr2=0x7f52a8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 --2- 172.21.15.112:0/4051036384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a81006a0 0x7f52a8100ac0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5298009900 tx=0x7f529802a410 comp rx=0 tx=0).stop 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 -- 172.21.15.112:0/4051036384 shutdown_connections 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 --2- 172.21.15.112:0/4051036384 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52a8102250 0x7f52a8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 --2- 172.21.15.112:0/4051036384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52a8101890 0x7f52a8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 --2- 172.21.15.112:0/4051036384 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a81006a0 0x7f52a8100ac0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 -- 172.21.15.112:0/4051036384 >> 172.21.15.112:0/4051036384 conn(0x7f52a80fbc30 msgr2=0x7f52a80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 -- 172.21.15.112:0/4051036384 shutdown_connections 2024-04-24T23:17:04.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 -- 172.21.15.112:0/4051036384 wait complete. 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 Processor -- start 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 -- start start 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52a81006a0 0x7f52a8196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.990+0000 7f52adaf5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52a8101890 0x7f52a8196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52adaf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8102250 0x7f52a819e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52adaf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52a810c450 con 0x7f52a8102250 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52adaf5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52a810c2d0 con 0x7f52a81006a0 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52adaf5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52a810c5d0 con 0x7f52a8101890 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8102250 0x7f52a819e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8102250 0x7f52a819e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48318/0 (socket says 172.21.15.112:48318) 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 -- 172.21.15.112:0/841092574 learned_addr learned my addr 172.21.15.112:0/841092574 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a77fe640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52a81006a0 0x7f52a8196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f529ffff640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52a8101890 0x7f52a8196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 -- 172.21.15.112:0/841092574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52a8101890 msgr2=0x7f52a8196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52a8101890 0x7f52a8196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 -- 172.21.15.112:0/841092574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52a81006a0 msgr2=0x7f52a8196aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52a81006a0 0x7f52a8196aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 -- 172.21.15.112:0/841092574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a819e6c0 con 0x7f52a8102250 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a7fff640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8102250 0x7f52a819e050 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f52a000eaa0 tx=0x7f52a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a57fa640 1 -- 172.21.15.112:0/841092574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a000cdb0 con 0x7f52a8102250 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a57fa640 1 -- 172.21.15.112:0/841092574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f52a0004b20 con 0x7f52a8102250 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52adaf5640 1 -- 172.21.15.112:0/841092574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a819e9b0 con 0x7f52a8102250 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52a57fa640 1 -- 172.21.15.112:0/841092574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0011760 con 0x7f52a8102250 2024-04-24T23:17:04.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.991+0000 7f52adaf5640 1 -- 172.21.15.112:0/841092574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a819eeb0 con 0x7f52a8102250 2024-04-24T23:17:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.993+0000 7f52a57fa640 1 -- 172.21.15.112:0/841092574 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f52a0004cd0 con 0x7f52a8102250 2024-04-24T23:17:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52b0801740 1 Processor -- start 2024-04-24T23:17:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52b0801740 1 -- start start 2024-04-24T23:17:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a57fa640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52940a2b90 0x7f52940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52b0801740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff411ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52b0801740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 0x55f3ff416b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52b0801740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff4171a0 0x55f3ff419630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52b0801740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff371670 con 0x55f3ff4171a0 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52b0801740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff319970 con 0x55f3ff411aa0 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52b0801740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff36d630 con 0x55f3ff412490 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f529ffff640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52940a2b90 0x7f52940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff4171a0 0x55f3ff419630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff4171a0 0x55f3ff419630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48326/0 (socket says 172.21.15.112:48326) 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 -- 172.21.15.112:0/3590012325 learned_addr learned my addr 172.21.15.112:0/3590012325 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a77fe640 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff411ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a57fa640 1 -- 172.21.15.112:0/841092574 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f52a00cf640 con 0x7f52a8102250 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f529ffff640 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 0x55f3ff416b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 -- 172.21.15.112:0/3590012325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 msgr2=0x55f3ff416b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 0x55f3ff416b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 -- 172.21.15.112:0/3590012325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 msgr2=0x55f3ff411ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff411ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 -- 172.21.15.112:0/3590012325 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ff33d400 con 0x55f3ff4171a0 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f529ffff640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52940a2b90 0x7f52940a5050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f52a8197fb0 tx=0x7f52900138e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.994+0000 7f52a7fff640 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff4171a0 0x55f3ff419630 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f52a00a41f0 tx=0x7f52a009b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d016bba7c8ae8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f529dffb640 1 -- 172.21.15.112:0/3590012325 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a001f040 con 0x55f3ff4171a0 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f529dffb640 1 -- 172.21.15.112:0/3590012325 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f52a009c1b0 con 0x55f3ff4171a0 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f529dffb640 1 -- 172.21.15.112:0/3590012325 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0098120 con 0x55f3ff4171a0 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 -- 172.21.15.112:0/3590012325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff4171a0 msgr2=0x55f3ff419630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff4171a0 0x55f3ff419630 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f52a00a41f0 tx=0x7f52a009b410 comp rx=0 tx=0).stop 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 -- 172.21.15.112:0/3590012325 shutdown_connections 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff4171a0 0x55f3ff419630 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 0x55f3ff416b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 --2- 172.21.15.112:0/3590012325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff411ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 -- 172.21.15.112:0/3590012325 >> 172.21.15.112:0/3590012325 conn(0x55f3ff409de0 msgr2=0x55f3ff408390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 -- 172.21.15.112:0/3590012325 shutdown_connections 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.995+0000 7f52b0801740 1 -- 172.21.15.112:0/3590012325 wait complete. 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 1 -- start start 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 1 Processor -- start 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff41b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff412490 0x55f3ff41bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff422c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff41a090 con 0x55f3ff411aa0 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff32d050 con 0x55f3ff412490 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f3ff41a210 con 0x55f3ff4171a0 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52b0801740 10 client.0 _ll_register_callbacks cb 0x55f3ff084be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff41b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff41b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48330/0 (socket says 172.21.15.112:48330) 2024-04-24T23:17:05.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a77fe640 1 -- 172.21.15.112:0/3877172196 learned_addr learned my addr 172.21.15.112:0/3877172196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f529ffff640 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff412490 0x55f3ff41bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a7fff640 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff422c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a77fe640 1 -- 172.21.15.112:0/3877172196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 msgr2=0x55f3ff422c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a77fe640 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff422c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a77fe640 1 -- 172.21.15.112:0/3877172196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff412490 msgr2=0x55f3ff41bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a77fe640 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff412490 0x55f3ff41bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f52a77fe640 1 -- 172.21.15.112:0/3877172196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ff2fedc0 con 0x55f3ff411aa0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.996+0000 7f529ffff640 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff412490 0x55f3ff41bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52a77fe640 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff41b6b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5298000c00 tx=0x7f5298031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3947a008359caf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f5286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f5286ffd640 1 -- 172.21.15.112:0/3877172196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298035040 con 0x55f3ff411aa0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f5286ffd640 1 -- 172.21.15.112:0/3877172196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f52980050c0 con 0x55f3ff411aa0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 1 -- 172.21.15.112:0/3877172196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a009c1b0 con 0x55f3ff411aa0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f5286ffd640 1 -- 172.21.15.112:0/3877172196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f52980052c0 con 0x55f3ff411aa0 2024-04-24T23:17:05.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 10 client.16179 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 1 -- 172.21.15.112:0/3877172196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3ff0853f0 con 0x55f3ff411aa0 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f5286ffd640 1 -- 172.21.15.112:0/3877172196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f52a009c1b0 con 0x55f3ff411aa0 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f5286ffd640 1 -- 172.21.15.112:0/3877172196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f5298003770 con 0x55f3ff411aa0 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 10 client.16179 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 10 client.16179 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 20 client.16179 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f3ff427540 0x55f3ff429a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 4 client.16179 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f52b0801740 1 -- 172.21.15.112:0/3877172196 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ff37ebb0 con 0x55f3ff427540 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.997+0000 7f529ffff640 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f3ff427540 0x55f3ff429a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.998+0000 7f5286ffd640 10 client.16179 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.998+0000 7f529ffff640 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f3ff427540 0x55f3ff429a00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1239320e0925f664 server_cookie=1bab7226baf9524f in_seq=0 out_seq=0 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f5286ffd640 1 -- 172.21.15.112:0/3877172196 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3ff37ebb0 con 0x55f3ff427540 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f5286ffd640 10 client.16179 handle_command_reply: tid=0 2024-04-24T23:17:05.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 client.16179 shutdown 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 -- 172.21.15.112:0/3877172196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 msgr2=0x55f3ff41b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff41b6b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5298000c00 tx=0x7f5298031ab0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 -- 172.21.15.112:0/3877172196 shutdown_connections 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f3ff427540 0x55f3ff429a00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff422c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff412490 0x55f3ff41bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f5286ffd640 0 client.16179 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 --2- 172.21.15.112:0/3877172196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff41b6b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 -- 172.21.15.112:0/3877172196 >> 172.21.15.112:0/3877172196 conn(0x55f3ff409de0 msgr2=0x55f3ff407b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 -- 172.21.15.112:0/3877172196 shutdown_connections 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 1 -- 172.21.15.112:0/3877172196 wait complete. 2024-04-24T23:17:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:04.999+0000 7f52b0801740 20 client.16179 trim_cache size 0 max 16384 2024-04-24T23:17:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52b0801740 1 Processor -- start 2024-04-24T23:17:05.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52b0801740 1 -- start start 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52b0801740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff407d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52b0801740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff412490 0x55f3ff422bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52b0801740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff42e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52b0801740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81a8410 con 0x55f3ff412490 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52b0801740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81aa070 con 0x55f3ff411aa0 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52b0801740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81a8ae0 con 0x55f3ff4171a0 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f529ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff412490 0x55f3ff422bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f529ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff412490 0x55f3ff422bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48336/0 (socket says 172.21.15.112:48336) 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f529ffff640 1 -- 172.21.15.112:0/209140335 learned_addr learned my addr 172.21.15.112:0/209140335 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f52a7fff640 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff42e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.012+0000 7f529ffff640 1 -- 172.21.15.112:0/209140335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 msgr2=0x55f3ff42e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52a77fe640 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff407d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f529ffff640 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff42e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f529ffff640 1 -- 172.21.15.112:0/209140335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 msgr2=0x55f3ff407d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f529ffff640 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff407d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f529ffff640 1 -- 172.21.15.112:0/209140335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ff14f7d0 con 0x55f3ff412490 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52a7fff640 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff42e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52a77fe640 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff407d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f529ffff640 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff412490 0x55f3ff422bf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f529000baf0 tx=0x7f5290005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a3f115f9c4bbeb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f529d7fa640 1 -- 172.21.15.112:0/209140335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290028d50 con 0x55f3ff412490 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f529d7fa640 1 -- 172.21.15.112:0/209140335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f529002d070 con 0x55f3ff412490 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f529d7fa640 1 -- 172.21.15.112:0/209140335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5290028880 con 0x55f3ff412490 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 -- 172.21.15.112:0/209140335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff412490 msgr2=0x55f3ff422bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff412490 0x55f3ff422bf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f529000baf0 tx=0x7f5290005ca0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 -- 172.21.15.112:0/209140335 shutdown_connections 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff4171a0 0x55f3ff42e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff412490 0x55f3ff422bf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 --2- 172.21.15.112:0/209140335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff411aa0 0x55f3ff407d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 -- 172.21.15.112:0/209140335 >> 172.21.15.112:0/209140335 conn(0x55f3ff409de0 msgr2=0x55f3ff413360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 -- 172.21.15.112:0/209140335 shutdown_connections 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.013+0000 7f52b0801740 1 -- 172.21.15.112:0/209140335 wait complete. 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 1 -- start start 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 1 Processor -- start 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff4205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 0x55f3ff420ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4171a0 0x55f3ff421af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81a98e0 con 0x55f3ff411aa0 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81a91e0 con 0x55f3ff4171a0 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52a81a63c0 con 0x55f3ff412490 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52b0801740 10 client.0 _ll_register_callbacks cb 0x7f5298009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff4205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52a77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff4205a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48344/0 (socket says 172.21.15.112:48344) 2024-04-24T23:17:05.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52a77fe640 1 -- 172.21.15.112:0/1359738702 learned_addr learned my addr 172.21.15.112:0/1359738702 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f529ffff640 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 0x55f3ff420ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52a7fff640 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4171a0 0x55f3ff421af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52a77fe640 1 -- 172.21.15.112:0/1359738702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 msgr2=0x55f3ff420ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52a77fe640 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 0x55f3ff420ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52a77fe640 1 -- 172.21.15.112:0/1359738702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4171a0 msgr2=0x55f3ff421af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.014+0000 7f52a77fe640 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4171a0 0x55f3ff421af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52a77fe640 1 -- 172.21.15.112:0/1359738702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ff4220e0 con 0x55f3ff411aa0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52a77fe640 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff4205a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5298003d80 tx=0x7f52980317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b2d688bf7bde8d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f5286ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f5286ffd640 1 -- 172.21.15.112:0/1359738702 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298035070 con 0x55f3ff411aa0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 1 -- 172.21.15.112:0/1359738702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f529002d070 con 0x55f3ff411aa0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f5286ffd640 1 -- 172.21.15.112:0/1359738702 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f52980037f0 con 0x55f3ff411aa0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f5286ffd640 1 -- 172.21.15.112:0/1359738702 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5298005d50 con 0x55f3ff411aa0 2024-04-24T23:17:05.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 10 client.16185 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 1 -- 172.21.15.112:0/1359738702 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f3ff48ddc0 con 0x55f3ff411aa0 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f5286ffd640 1 -- 172.21.15.112:0/1359738702 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f529002d070 con 0x55f3ff411aa0 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f5286ffd640 1 -- 172.21.15.112:0/1359738702 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f52980049b0 con 0x55f3ff411aa0 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 10 client.16185 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 10 client.16185 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 20 client.16185 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f3ff48e6b0 0x55f3ff490ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 4 client.16185 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.015+0000 7f52b0801740 1 -- 172.21.15.112:0/1359738702 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3ff378f00 con 0x55f3ff48e6b0 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.016+0000 7f529ffff640 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f3ff48e6b0 0x55f3ff490ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.016+0000 7f5286ffd640 10 client.16185 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.016+0000 7f529ffff640 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f3ff48e6b0 0x55f3ff490ad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=344d503bae685ca5 server_cookie=642b179081857043 in_seq=0 out_seq=0 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.016+0000 7f5286ffd640 1 -- 172.21.15.112:0/1359738702 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3ff378f00 con 0x55f3ff48e6b0 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.016+0000 7f5286ffd640 10 client.16185 handle_command_reply: tid=0 2024-04-24T23:17:05.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.016+0000 7f52b0801740 1 client.16185 shutdown 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 -- 172.21.15.112:0/1359738702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 msgr2=0x55f3ff4205a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff4205a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5298003d80 tx=0x7f52980317b0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 -- 172.21.15.112:0/1359738702 shutdown_connections 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x55f3ff48e6b0 0x55f3ff490ad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55f3ff4171a0 0x55f3ff421af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f3ff412490 0x55f3ff420ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f5286ffd640 0 client.16185 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 --2- 172.21.15.112:0/1359738702 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f3ff411aa0 0x55f3ff4205a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 -- 172.21.15.112:0/1359738702 >> 172.21.15.112:0/1359738702 conn(0x55f3ff409de0 msgr2=0x55f3ff41aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 -- 172.21.15.112:0/1359738702 shutdown_connections 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 1 -- 172.21.15.112:0/1359738702 wait complete. 2024-04-24T23:17:05.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52b0801740 20 client.16185 trim_cache size 0 max 16384 2024-04-24T23:17:05.026 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:05.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 -- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52940a2b90 msgr2=0x7f52940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52940a2b90 0x7f52940a5050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f52a8197fb0 tx=0x7f52900138e0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 -- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8102250 msgr2=0x7f52a819e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8102250 0x7f52a819e050 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f52a000eaa0 tx=0x7f52a000ef60 comp rx=0 tx=0).stop 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 -- 172.21.15.112:0/841092574 shutdown_connections 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f52940a2b90 0x7f52940a5050 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52a8102250 0x7f52a819e050 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52a8101890 0x7f52a8196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 --2- 172.21.15.112:0/841092574 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52a81006a0 0x7f52a8196aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 -- 172.21.15.112:0/841092574 >> 172.21.15.112:0/841092574 conn(0x7f52a80fbc30 msgr2=0x7f52a8104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 -- 172.21.15.112:0/841092574 shutdown_connections 2024-04-24T23:17:05.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.017+0000 7f52adaf5640 1 -- 172.21.15.112:0/841092574 wait complete. 2024-04-24T23:17:05.030 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T23:17:05.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 -- 172.21.15.112:0/2861591041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610100680 msgr2=0x7f3610100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 --2- 172.21.15.112:0/2861591041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610100680 0x7f3610100aa0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3604009900 tx=0x7f360402a410 comp rx=0 tx=0).stop 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 -- 172.21.15.112:0/2861591041 shutdown_connections 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 --2- 172.21.15.112:0/2861591041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3610102230 0x7f3610108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 --2- 172.21.15.112:0/2861591041 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3610101870 0x7f3610101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 --2- 172.21.15.112:0/2861591041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610100680 0x7f3610100aa0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 -- 172.21.15.112:0/2861591041 >> 172.21.15.112:0/2861591041 conn(0x7f36100fbbf0 msgr2=0x7f36100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 -- 172.21.15.112:0/2861591041 shutdown_connections 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.127+0000 7f3616c83640 1 -- 172.21.15.112:0/2861591041 wait complete. 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3616c83640 1 Processor -- start 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3616c83640 1 -- start start 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3616c83640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3610100680 0x7f3610196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3616c83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610101870 0x7f3610197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3616c83640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3610102230 0x7f361019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3616c83640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f361010c430 con 0x7f3610101870 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3616c83640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f361010c2b0 con 0x7f3610100680 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3616c83640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f361010c5b0 con 0x7f3610102230 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f360ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610101870 0x7f3610197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f360ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610101870 0x7f3610197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48350/0 (socket says 172.21.15.112:48350) 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f360ffff640 1 -- 172.21.15.112:0/506780978 learned_addr learned my addr 172.21.15.112:0/506780978 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3614a1f640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3610100680 0x7f3610196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3615220640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3610102230 0x7f361019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f360ffff640 1 -- 172.21.15.112:0/506780978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3610102230 msgr2=0x7f361019e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f360ffff640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3610102230 0x7f361019e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f360ffff640 1 -- 172.21.15.112:0/506780978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3610100680 msgr2=0x7f3610196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f360ffff640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3610100680 0x7f3610196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f360ffff640 1 -- 172.21.15.112:0/506780978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361019e700 con 0x7f3610101870 2024-04-24T23:17:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3614a1f640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3610100680 0x7f3610196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.128+0000 7f3615220640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3610102230 0x7f361019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.129+0000 7f360ffff640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610101870 0x7f3610197020 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009970 tx=0x7f35fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.129+0000 7f360dffb640 1 -- 172.21.15.112:0/506780978 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc00d020 con 0x7f3610101870 2024-04-24T23:17:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.129+0000 7f360dffb640 1 -- 172.21.15.112:0/506780978 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f35fc007d60 con 0x7f3610101870 2024-04-24T23:17:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.129+0000 7f3616c83640 1 -- 172.21.15.112:0/506780978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3610068930 con 0x7f3610101870 2024-04-24T23:17:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.129+0000 7f360dffb640 1 -- 172.21.15.112:0/506780978 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc00cdc0 con 0x7f3610101870 2024-04-24T23:17:05.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.129+0000 7f3616c83640 1 -- 172.21.15.112:0/506780978 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3610068e80 con 0x7f3610101870 2024-04-24T23:17:05.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f360dffb640 1 -- 172.21.15.112:0/506780978 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f35fc00d1c0 con 0x7f3610101870 2024-04-24T23:17:05.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f361998a740 1 Processor -- start 2024-04-24T23:17:05.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f361998a740 1 -- start start 2024-04-24T23:17:05.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f361998a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce03650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f361998a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce08720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f361998a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce0b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f361998a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56193cd67670 con 0x56193ce03c20 2024-04-24T23:17:05.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f361998a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56193cd0f970 con 0x56193ce03230 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f361998a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56193cd63630 con 0x56193ce08d90 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f3614a1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce08720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f3614a1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce08720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48354/0 (socket says 172.21.15.112:48354) 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f3614a1f640 1 -- 172.21.15.112:0/3992406123 learned_addr learned my addr 172.21.15.112:0/3992406123 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.132+0000 7f3615220640 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce03650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f360ffff640 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce0b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f360dffb640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f36000a2b90 0x7f36000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f3614a1f640 1 -- 172.21.15.112:0/3992406123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 msgr2=0x56193ce0b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce0b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f3614a1f640 1 -- 172.21.15.112:0/3992406123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 msgr2=0x56193ce03650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce03650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f3614a1f640 1 -- 172.21.15.112:0/3992406123 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56193cd33400 con 0x56193ce03c20 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f3615220640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f36000a2b90 0x7f36000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f360dffb640 1 -- 172.21.15.112:0/506780978 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f35fc0d0970 con 0x7f3610101870 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce08720 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3604000c00 tx=0x7f3604032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc751d68e7eb1072 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f35faffd640 1 -- 172.21.15.112:0/3992406123 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360403f070 con 0x56193ce03c20 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f35faffd640 1 -- 172.21.15.112:0/3992406123 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3604031a00 con 0x56193ce03c20 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f35faffd640 1 -- 172.21.15.112:0/3992406123 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604035070 con 0x56193ce03c20 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.133+0000 7f3615220640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f36000a2b90 0x7f36000a5050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x56193ce04480 tx=0x7f3608004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 -- 172.21.15.112:0/3992406123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 msgr2=0x56193ce08720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce08720 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3604000c00 tx=0x7f3604032dd0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 -- 172.21.15.112:0/3992406123 shutdown_connections 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce0b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce08720 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 --2- 172.21.15.112:0/3992406123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce03650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 -- 172.21.15.112:0/3992406123 >> 172.21.15.112:0/3992406123 conn(0x56193cdfe8a0 msgr2=0x56193cdfeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 -- 172.21.15.112:0/3992406123 shutdown_connections 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 -- 172.21.15.112:0/3992406123 wait complete. 2024-04-24T23:17:05.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 -- start start 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.134+0000 7f361998a740 1 Processor -- start 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce0d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce0d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce0d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56193ce0bce0 con 0x56193ce03c20 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x56193cd23050 con 0x56193ce03230 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce0d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48358/0 (socket says 172.21.15.112:48358) 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56193ce0be60 con 0x56193ce08d90 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 -- 172.21.15.112:0/967369880 learned_addr learned my addr 172.21.15.112:0/967369880 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 10 client.0 _ll_register_callbacks cb 0x56193ca7abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f3614a1f640 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce0d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f3615220640 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 -- 172.21.15.112:0/967369880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 msgr2=0x56193ce148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 -- 172.21.15.112:0/967369880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 msgr2=0x56193ce0d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce0d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 -- 172.21.15.112:0/967369880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56193ccf4dc0 con 0x56193ce03c20 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f360ffff640 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce0d890 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0ce080 tx=0x7f35fc09cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d88b31d3d650d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f35effff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f35effff640 1 -- 172.21.15.112:0/967369880 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc09c430 con 0x56193ce03c20 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f35effff640 1 -- 172.21.15.112:0/967369880 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f35fc007860 con 0x56193ce03c20 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3604031a00 con 0x56193ce03c20 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f35effff640 1 -- 172.21.15.112:0/967369880 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f35fc0a5a50 con 0x56193ce03c20 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.135+0000 7f361998a740 10 client.16197 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56193ca7b3f0 con 0x56193ce03c20 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f35effff640 1 -- 172.21.15.112:0/967369880 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3604031a00 con 0x56193ce03c20 2024-04-24T23:17:05.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f35effff640 1 -- 172.21.15.112:0/967369880 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f35fc09bca0 con 0x56193ce03c20 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f361998a740 10 client.16197 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f361998a740 10 client.16197 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f361998a740 20 client.16197 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f361998a740 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x56193ce19100 0x56193ce1b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f361998a740 4 client.16197 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56193cd74bb0 con 0x56193ce19100 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.136+0000 7f3614a1f640 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x56193ce19100 0x56193ce1b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.137+0000 7f35effff640 10 client.16197 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.137+0000 7f3614a1f640 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x56193ce19100 0x56193ce1b5c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f46d3abc8ca53cd8 server_cookie=daa3695b5229b7b5 in_seq=0 out_seq=0 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.137+0000 7f35effff640 1 -- 172.21.15.112:0/967369880 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56193cd74bb0 con 0x56193ce19100 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.137+0000 7f35effff640 10 client.16197 handle_command_reply: tid=0 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.137+0000 7f361998a740 1 client.16197 shutdown 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 msgr2=0x56193ce0d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce0d890 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0ce080 tx=0x7f35fc09cbb0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 shutdown_connections 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x56193ce19100 0x56193ce1b5c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce0d890 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 --2- 172.21.15.112:0/967369880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce0d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f35effff640 0 client.16197 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 >> 172.21.15.112:0/967369880 conn(0x56193cdfe8a0 msgr2=0x56193cdfdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 shutdown_connections 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 1 -- 172.21.15.112:0/967369880 wait complete. 2024-04-24T23:17:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.138+0000 7f361998a740 20 client.16197 trim_cache size 0 max 16384 2024-04-24T23:17:05.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f361998a740 1 Processor -- start 2024-04-24T23:17:05.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f361998a740 1 -- start start 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f361998a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03230 0x56193ce017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f361998a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03c20 0x56193ce14890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f361998a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce01cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f361998a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36101a83a0 con 0x56193ce03230 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f361998a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f36101aa000 con 0x56193ce03c20 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f361998a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36101a8a70 con 0x56193ce08d90 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f3614a1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03230 0x56193ce017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f3614a1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03230 0x56193ce017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48366/0 (socket says 172.21.15.112:48366) 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f3614a1f640 1 -- 172.21.15.112:0/3275107549 learned_addr learned my addr 172.21.15.112:0/3275107549 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f3614a1f640 1 -- 172.21.15.112:0/3275107549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 msgr2=0x56193ce01cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce01cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f360ffff640 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03c20 0x56193ce14890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f3614a1f640 1 -- 172.21.15.112:0/3275107549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03c20 msgr2=0x56193ce14890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03c20 0x56193ce14890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f3614a1f640 1 -- 172.21.15.112:0/3275107549 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56193cb457d0 con 0x56193ce03230 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.151+0000 7f360ffff640 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03c20 0x56193ce14890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03230 0x56193ce017b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f36040096c0 tx=0x7f36040058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce9e073c97805e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f35fa7fc640 1 -- 172.21.15.112:0/3275107549 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360403f040 con 0x56193ce03230 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f35fa7fc640 1 -- 172.21.15.112:0/3275107549 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f36040044a0 con 0x56193ce03230 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f35fa7fc640 1 -- 172.21.15.112:0/3275107549 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604035040 con 0x56193ce03230 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 -- 172.21.15.112:0/3275107549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03230 msgr2=0x56193ce017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03230 0x56193ce017b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f36040096c0 tx=0x7f36040058b0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 -- 172.21.15.112:0/3275107549 shutdown_connections 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce01cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03c20 0x56193ce14890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 --2- 172.21.15.112:0/3275107549 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03230 0x56193ce017b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 -- 172.21.15.112:0/3275107549 >> 172.21.15.112:0/3275107549 conn(0x56193cdfe8a0 msgr2=0x56193cdfdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 -- 172.21.15.112:0/3275107549 shutdown_connections 2024-04-24T23:17:05.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.152+0000 7f361998a740 1 -- 172.21.15.112:0/3275107549 wait complete. 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 1 -- start start 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 1 Processor -- start 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce11e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce12380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36101a9870 con 0x56193ce03c20 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f36101a9170 con 0x56193ce03230 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36101a6350 con 0x56193ce08d90 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f361998a740 10 client.0 _ll_register_callbacks cb 0x7f35fc099540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f360ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce12380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f360ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce12380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48378/0 (socket says 172.21.15.112:48378) 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f360ffff640 1 -- 172.21.15.112:0/3261414788 learned_addr learned my addr 172.21.15.112:0/3261414788 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce11e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f3615220640 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f360ffff640 1 -- 172.21.15.112:0/3261414788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 msgr2=0x56193ce132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f360ffff640 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce132f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f360ffff640 1 -- 172.21.15.112:0/3261414788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 msgr2=0x56193ce11e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f360ffff640 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce11e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.153+0000 7f360ffff640 1 -- 172.21.15.112:0/3261414788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56193ce138e0 con 0x56193ce03c20 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f360ffff640 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce12380 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0ce3f0 tx=0x7f35fc09b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901c761938ab4d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f35effff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f35effff640 1 -- 172.21.15.112:0/3261414788 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc09c920 con 0x56193ce03c20 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f35effff640 1 -- 172.21.15.112:0/3261414788 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f35fc098b30 con 0x56193ce03c20 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 1 -- 172.21.15.112:0/3261414788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36040044a0 con 0x56193ce03c20 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f35effff640 1 -- 172.21.15.112:0/3261414788 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f35fc0d6030 con 0x56193ce03c20 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 10 client.16203 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 1 -- 172.21.15.112:0/3261414788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56193ce14bc0 con 0x56193ce03c20 2024-04-24T23:17:05.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f35effff640 1 -- 172.21.15.112:0/3261414788 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f36040044a0 con 0x56193ce03c20 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f35effff640 1 -- 172.21.15.112:0/3261414788 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f35fc0873e0 con 0x56193ce03c20 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 10 client.16203 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 10 client.16203 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 20 client.16203 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x56193ce7fce0 0x56193ce82100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 4 client.16203 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f361998a740 1 -- 172.21.15.112:0/3261414788 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56193cd6ef00 con 0x56193ce7fce0 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.154+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x56193ce7fce0 0x56193ce82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f35effff640 10 client.16203 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f3614a1f640 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x56193ce7fce0 0x56193ce82100 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53fd1f234e1b8783 server_cookie=4880f4b2727f6411 in_seq=0 out_seq=0 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f35effff640 1 -- 172.21.15.112:0/3261414788 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56193cd6ef00 con 0x56193ce7fce0 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f35effff640 10 client.16203 handle_command_reply: tid=0 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 client.16203 shutdown 2024-04-24T23:17:05.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 -- 172.21.15.112:0/3261414788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 msgr2=0x56193ce12380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce12380 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0ce3f0 tx=0x7f35fc09b2f0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 -- 172.21.15.112:0/3261414788 shutdown_connections 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x56193ce7fce0 0x56193ce82100 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56193ce08d90 0x56193ce132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f35effff640 0 client.16203 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56193ce03c20 0x56193ce12380 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 --2- 172.21.15.112:0/3261414788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56193ce03230 0x56193ce11e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 -- 172.21.15.112:0/3261414788 >> 172.21.15.112:0/3261414788 conn(0x56193cdfe8a0 msgr2=0x56193ce046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 -- 172.21.15.112:0/3261414788 shutdown_connections 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 1 -- 172.21.15.112:0/3261414788 wait complete. 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.155+0000 7f361998a740 20 client.16203 trim_cache size 0 max 16384 2024-04-24T23:17:05.164 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 -- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f36000a2b90 msgr2=0x7f36000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f36000a2b90 0x7f36000a5050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x56193ce04480 tx=0x7f3608004a40 comp rx=0 tx=0).stop 2024-04-24T23:17:05.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 -- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610101870 msgr2=0x7f3610197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610101870 0x7f3610197020 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009970 tx=0x7f35fc009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 -- 172.21.15.112:0/506780978 shutdown_connections 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f36000a2b90 0x7f36000a5050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3610102230 0x7f361019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3610101870 0x7f3610197020 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 --2- 172.21.15.112:0/506780978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3610100680 0x7f3610196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 -- 172.21.15.112:0/506780978 >> 172.21.15.112:0/506780978 conn(0x7f36100fbbf0 msgr2=0x7f3610104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 -- 172.21.15.112:0/506780978 shutdown_connections 2024-04-24T23:17:05.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.156+0000 7f3616c83640 1 -- 172.21.15.112:0/506780978 wait complete. 2024-04-24T23:17:05.167 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:04 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3699925912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:05.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:17:05.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:04 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3699925912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:05.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.288+0000 7f31897ce640 1 -- 172.21.15.112:0/312091857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184072c30 msgr2=0x7f3184073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.288+0000 7f31897ce640 1 --2- 172.21.15.112:0/312091857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184072c30 0x7f3184073050 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3174009900 tx=0x7f317402a420 comp rx=0 tx=0).stop 2024-04-24T23:17:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.293+0000 7f31897ce640 1 -- 172.21.15.112:0/312091857 shutdown_connections 2024-04-24T23:17:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.293+0000 7f31897ce640 1 --2- 172.21.15.112:0/312091857 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3184109820 0x7f318410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.293+0000 7f31897ce640 1 --2- 172.21.15.112:0/312091857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3184108e80 0x7f31841092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.293+0000 7f31897ce640 1 --2- 172.21.15.112:0/312091857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184072c30 0x7f3184073050 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.293+0000 7f31897ce640 1 -- 172.21.15.112:0/312091857 >> 172.21.15.112:0/312091857 conn(0x7f318406e650 msgr2=0x7f3184070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.293+0000 7f31897ce640 1 -- 172.21.15.112:0/312091857 shutdown_connections 2024-04-24T23:17:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.293+0000 7f31897ce640 1 -- 172.21.15.112:0/312091857 wait complete. 2024-04-24T23:17:05.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.294+0000 7f31897ce640 1 Processor -- start 2024-04-24T23:17:05.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.295+0000 7f31897ce640 1 -- start start 2024-04-24T23:17:05.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.295+0000 7f31897ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3184072c30 0x7f31841a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.295+0000 7f31897ce640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3184108e80 0x7f31841a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.295+0000 7f31897ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184109820 0x7f31841adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.295+0000 7f31897ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f318411c310 con 0x7f3184109820 2024-04-24T23:17:05.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.296+0000 7f31897ce640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f318411c190 con 0x7f3184108e80 2024-04-24T23:17:05.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.296+0000 7f31897ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f318411c490 con 0x7f3184072c30 2024-04-24T23:17:05.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.297+0000 7f31827fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3184108e80 0x7f31841a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.297+0000 7f31827fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3184108e80 0x7f31841a6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50218/0 (socket says 172.21.15.112:50218) 2024-04-24T23:17:05.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.297+0000 7f31827fc640 1 -- 172.21.15.112:0/826576391 learned_addr learned my addr 172.21.15.112:0/826576391 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.297+0000 7f31837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184109820 0x7f31841adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.297+0000 7f3182ffd640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3184072c30 0x7f31841a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.298+0000 7f31837fe640 1 -- 172.21.15.112:0/826576391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3184072c30 msgr2=0x7f31841a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.298+0000 7f31837fe640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3184072c30 0x7f31841a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.298+0000 7f31837fe640 1 -- 172.21.15.112:0/826576391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3184108e80 msgr2=0x7f31841a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.298+0000 7f31837fe640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3184108e80 0x7f31841a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.298+0000 7f31837fe640 1 -- 172.21.15.112:0/826576391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318410ffa0 con 0x7f3184109820 2024-04-24T23:17:05.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.298+0000 7f31837fe640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184109820 0x7f31841adea0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f317c00c9e0 tx=0x7f317c00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.299+0000 7f316bfff640 1 -- 172.21.15.112:0/826576391 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c004d20 con 0x7f3184109820 2024-04-24T23:17:05.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.299+0000 7f31897ce640 1 -- 172.21.15.112:0/826576391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3184110290 con 0x7f3184109820 2024-04-24T23:17:05.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.299+0000 7f31897ce640 1 -- 172.21.15.112:0/826576391 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3184110860 con 0x7f3184109820 2024-04-24T23:17:05.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.299+0000 7f316bfff640 1 -- 172.21.15.112:0/826576391 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f317c005020 con 0x7f3184109820 2024-04-24T23:17:05.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.299+0000 7f316bfff640 1 -- 172.21.15.112:0/826576391 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c00f780 con 0x7f3184109820 2024-04-24T23:17:05.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.305+0000 7f318c5b0740 1 Processor -- start 2024-04-24T23:17:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.306+0000 7f318c5b0740 1 -- start start 2024-04-24T23:17:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.306+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccd3d0 0x556a75ccd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.306+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75ccddc0 0x556a75cd28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.306+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cd2f30 0x556a75cd53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.306+0000 7f318c5b0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556a75c31670 con 0x556a75ccddc0 2024-04-24T23:17:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.306+0000 7f318c5b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556a75bd9970 con 0x556a75cd2f30 2024-04-24T23:17:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.306+0000 7f318c5b0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556a75c2d630 con 0x556a75ccd3d0 2024-04-24T23:17:05.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.306+0000 7f316bfff640 1 -- 172.21.15.112:0/826576391 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f317c007aa0 con 0x7f3184109820 2024-04-24T23:17:05.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.307+0000 7f316bfff640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f316c0a2d90 0x7f316c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.307+0000 7f316bfff640 1 -- 172.21.15.112:0/826576391 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f317c0cf450 con 0x7f3184109820 2024-04-24T23:17:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.308+0000 7f31827fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccd3d0 0x556a75ccd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.308+0000 7f31827fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cd2f30 0x556a75cd53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.308+0000 7f31837fe640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f316c0a2d90 0x7f316c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.309+0000 7f31837fe640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f316c0a2d90 0x7f316c0a5250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f317c0a3fa0 tx=0x7f317c005ae0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.310+0000 7f31827fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccd3d0 0x556a75ccd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36818/0 (socket says 172.21.15.112:36818) 2024-04-24T23:17:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.310+0000 7f31827fc640 1 -- 172.21.15.112:0/533015575 learned_addr learned my addr 172.21.15.112:0/533015575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.310+0000 7f3182ffd640 1 --2- 172.21.15.112:0/533015575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75ccddc0 0x556a75cd28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.313+0000 7f31827fc640 1 -- 172.21.15.112:0/533015575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cd2f30 msgr2=0x556a75cd53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.313+0000 7f31827fc640 1 --2- 172.21.15.112:0/533015575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cd2f30 0x556a75cd53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.313+0000 7f31827fc640 1 -- 172.21.15.112:0/533015575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75ccddc0 msgr2=0x556a75cd28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.313+0000 7f31827fc640 1 --2- 172.21.15.112:0/533015575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75ccddc0 0x556a75cd28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.313+0000 7f31827fc640 1 -- 172.21.15.112:0/533015575 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a75bfd400 con 0x556a75ccd3d0 2024-04-24T23:17:05.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.314+0000 7f31827fc640 1 --2- 172.21.15.112:0/533015575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccd3d0 0x556a75ccd7f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3178017600 tx=0x7f3178009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fab16c6541c51920 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:04 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3699925912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.319+0000 7f31697fa640 1 -- 172.21.15.112:0/533015575 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317801d070 con 0x556a75ccd3d0 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.319+0000 7f31697fa640 1 -- 172.21.15.112:0/533015575 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3178016a80 con 0x556a75ccd3d0 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.320+0000 7f318c5b0740 1 -- 172.21.15.112:0/533015575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccd3d0 msgr2=0x556a75ccd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.320+0000 7f318c5b0740 1 --2- 172.21.15.112:0/533015575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccd3d0 0x556a75ccd7f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3178017600 tx=0x7f3178009190 comp rx=0 tx=0).stop 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.321+0000 7f318c5b0740 1 -- 172.21.15.112:0/533015575 shutdown_connections 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.321+0000 7f318c5b0740 1 --2- 172.21.15.112:0/533015575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cd2f30 0x556a75cd53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.321+0000 7f318c5b0740 1 --2- 172.21.15.112:0/533015575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75ccddc0 0x556a75cd28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.321+0000 7f318c5b0740 1 --2- 172.21.15.112:0/533015575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccd3d0 0x556a75ccd7f0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3178017600 tx=0x7f3178009190 comp rx=0 tx=0).stop 2024-04-24T23:17:05.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.321+0000 7f318c5b0740 1 -- 172.21.15.112:0/533015575 >> 172.21.15.112:0/533015575 conn(0x556a75cc8a40 msgr2=0x556a75cc8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.324+0000 7f318c5b0740 1 -- 172.21.15.112:0/533015575 shutdown_connections 2024-04-24T23:17:05.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.325+0000 7f318c5b0740 1 -- 172.21.15.112:0/533015575 wait complete. 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 1 -- start start 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 1 Processor -- start 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 0x556a75cd7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cd7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cdcce0 0x556a75cdf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556a75cd5f30 con 0x556a75cd2f30 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x556a75bed050 con 0x556a75ccddc0 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556a75cd60b0 con 0x556a75cdcce0 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.326+0000 7f318c5b0740 10 client.0 _ll_register_callbacks cb 0x556a75944be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.328+0000 7f31827fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 0x556a75cd7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.328+0000 7f31827fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cdcce0 0x556a75cdf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.329+0000 7f3182ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cd7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.329+0000 7f3182ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cd7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48408/0 (socket says 172.21.15.112:48408) 2024-04-24T23:17:05.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.329+0000 7f3182ffd640 1 -- 172.21.15.112:0/3135276120 learned_addr learned my addr 172.21.15.112:0/3135276120 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.331+0000 7f31827fc640 1 -- 172.21.15.112:0/3135276120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 msgr2=0x556a75cd7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.331+0000 7f31827fc640 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 0x556a75cd7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.331+0000 7f31827fc640 1 -- 172.21.15.112:0/3135276120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 msgr2=0x556a75cd7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.331+0000 7f31827fc640 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cd7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.331+0000 7f31827fc640 1 -- 172.21.15.112:0/3135276120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a75bbedc0 con 0x556a75cdcce0 2024-04-24T23:17:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.331+0000 7f3182ffd640 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cd7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.332+0000 7f31827fc640 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cdcce0 0x556a75cdf0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x556a75cc9940 tx=0x7f3178009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7533833abd0ce085 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.332+0000 7f3166ff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-24T23:17:05.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.332+0000 7f3166ff5640 1 -- 172.21.15.112:0/3135276120 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317802c060 con 0x556a75cdcce0 2024-04-24T23:17:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.332+0000 7f318c5b0740 1 -- 172.21.15.112:0/3135276120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3178016a80 con 0x556a75cdcce0 2024-04-24T23:17:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.333+0000 7f3166ff5640 1 -- 172.21.15.112:0/3135276120 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f3178004040 con 0x556a75cdcce0 2024-04-24T23:17:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.333+0000 7f3166ff5640 1 -- 172.21.15.112:0/3135276120 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f317801d040 con 0x556a75cdcce0 2024-04-24T23:17:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.333+0000 7f318c5b0740 10 client.25531 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.333+0000 7f318c5b0740 1 -- 172.21.15.112:0/3135276120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556a759453f0 con 0x556a75cdcce0 2024-04-24T23:17:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.334+0000 7f3166ff5640 1 -- 172.21.15.112:0/3135276120 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3178016a80 con 0x556a75cdcce0 2024-04-24T23:17:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.335+0000 7f3166ff5640 1 -- 172.21.15.112:0/3135276120 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f3178015050 con 0x556a75cdcce0 2024-04-24T23:17:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.335+0000 7f318c5b0740 10 client.25531 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.335+0000 7f318c5b0740 10 client.25531 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:17:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.335+0000 7f318c5b0740 20 client.25531 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.335+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x556a75ce3cd0 0x556a75ce6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.335+0000 7f318c5b0740 4 client.25531 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.335+0000 7f318c5b0740 1 -- 172.21.15.112:0/3135276120 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a75c3ebb0 con 0x556a75ce3cd0 2024-04-24T23:17:05.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.337+0000 7f3182ffd640 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x556a75ce3cd0 0x556a75ce6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.342+0000 7f3182ffd640 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x556a75ce3cd0 0x556a75ce6130 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=25ae57c91b51e50d server_cookie=de37bf43c3008a16 in_seq=0 out_seq=0 2024-04-24T23:17:05.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.342+0000 7f3166ff5640 10 client.25531 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:17:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.344+0000 7f3166ff5640 1 -- 172.21.15.112:0/3135276120 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a75c3ebb0 con 0x556a75ce3cd0 2024-04-24T23:17:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.344+0000 7f3166ff5640 10 client.25531 handle_command_reply: tid=0 2024-04-24T23:17:05.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.344+0000 7f318c5b0740 1 client.25531 shutdown 2024-04-24T23:17:05.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f318c5b0740 1 -- 172.21.15.112:0/3135276120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cdcce0 msgr2=0x556a75cdf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cdcce0 0x556a75cdf0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x556a75cc9940 tx=0x7f3178009a20 comp rx=0 tx=0).stop 2024-04-24T23:17:05.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f318c5b0740 1 -- 172.21.15.112:0/3135276120 shutdown_connections 2024-04-24T23:17:05.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x556a75ce3cd0 0x556a75ce6130 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cdcce0 0x556a75cdf0d0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x556a75cc9940 tx=0x7f3178009a20 comp rx=0 tx=0).stop 2024-04-24T23:17:05.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cd7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3135276120 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 0x556a75cd7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f318c5b0740 1 -- 172.21.15.112:0/3135276120 >> 172.21.15.112:0/3135276120 conn(0x556a75cc8a40 msgr2=0x556a75cc7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.346+0000 7f3166ff5640 0 client.25531 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:17:05.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.349+0000 7f318c5b0740 1 -- 172.21.15.112:0/3135276120 shutdown_connections 2024-04-24T23:17:05.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.349+0000 7f318c5b0740 1 -- 172.21.15.112:0/3135276120 wait complete. 2024-04-24T23:17:05.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.350+0000 7f318c5b0740 20 client.25531 trim_cache size 0 max 16384 2024-04-24T23:17:05.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.378+0000 7f318c5b0740 1 Processor -- start 2024-04-24T23:17:05.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.379+0000 7f318c5b0740 1 -- start start 2024-04-24T23:17:05.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.379+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 0x556a75ce5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.379+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75ccc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cc7330 0x556a75cc77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f318c5b0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f317000fe10 con 0x556a75cd2f30 2024-04-24T23:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f318c5b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3170011a70 con 0x556a75ccddc0 2024-04-24T23:17:05.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f318c5b0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31700104e0 con 0x556a75cc7330 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cc7330 0x556a75cc77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cc7330 0x556a75cc77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36828/0 (socket says 172.21.15.112:36828) 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 -- 172.21.15.112:0/3770401031 learned_addr learned my addr 172.21.15.112:0/3770401031 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 0x556a75ce5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f3182ffd640 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75ccc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 -- 172.21.15.112:0/3770401031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 msgr2=0x556a75ce5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 0x556a75ce5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 -- 172.21.15.112:0/3770401031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 msgr2=0x556a75ccc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75ccc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f31827fc640 1 -- 172.21.15.112:0/3770401031 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a75cdf440 con 0x556a75cc7330 2024-04-24T23:17:05.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.380+0000 7f3182ffd640 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75ccc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:05.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.387+0000 7f31827fc640 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cc7330 0x556a75cc77e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f31780099f0 tx=0x7f3178009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a63dd94fb4173731 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.390+0000 7f3168ff9640 1 -- 172.21.15.112:0/3770401031 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317802c030 con 0x556a75cc7330 2024-04-24T23:17:05.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.390+0000 7f3168ff9640 1 -- 172.21.15.112:0/3770401031 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f31780183f0 con 0x556a75cc7330 2024-04-24T23:17:05.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.391+0000 7f318c5b0740 1 -- 172.21.15.112:0/3770401031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cc7330 msgr2=0x556a75cc77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.391+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cc7330 0x556a75cc77e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f31780099f0 tx=0x7f3178009a20 comp rx=0 tx=0).stop 2024-04-24T23:17:05.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.392+0000 7f318c5b0740 1 -- 172.21.15.112:0/3770401031 shutdown_connections 2024-04-24T23:17:05.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.392+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75cc7330 0x556a75cc77e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.392+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75ccc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.392+0000 7f318c5b0740 1 --2- 172.21.15.112:0/3770401031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75ccddc0 0x556a75ce5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.392+0000 7f318c5b0740 1 -- 172.21.15.112:0/3770401031 >> 172.21.15.112:0/3770401031 conn(0x556a75cc8a40 msgr2=0x556a75ccb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.392+0000 7f318c5b0740 1 -- 172.21.15.112:0/3770401031 shutdown_connections 2024-04-24T23:17:05.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.392+0000 7f318c5b0740 1 -- 172.21.15.112:0/3770401031 wait complete. 2024-04-24T23:17:05.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 1 -- start start 2024-04-24T23:17:05.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 1 Processor -- start 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cc7330 0x556a75cf4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccddc0 0x556a75cf50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cf6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31700112e0 con 0x556a75cd2f30 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3170010be0 con 0x556a75cc7330 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f317000ddc0 con 0x556a75ccddc0 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.393+0000 7f318c5b0740 10 client.0 _ll_register_callbacks cb 0x7f317801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.394+0000 7f3182ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccddc0 0x556a75cf50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.394+0000 7f3182ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccddc0 0x556a75cf50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36842/0 (socket says 172.21.15.112:36842) 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.394+0000 7f3182ffd640 1 -- 172.21.15.112:0/1120088646 learned_addr learned my addr 172.21.15.112:0/1120088646 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.394+0000 7f31827fc640 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cc7330 0x556a75cf4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.395+0000 7f31827fc640 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.395+0000 7f3182ffd640 1 -- 172.21.15.112:0/1120088646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cc7330 msgr2=0x556a75cf4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.395+0000 7f3182ffd640 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cc7330 0x556a75cf4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.395+0000 7f3182ffd640 1 -- 172.21.15.112:0/1120088646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 msgr2=0x556a75cf6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.395+0000 7f3182ffd640 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cf6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.395+0000 7f3182ffd640 1 -- 172.21.15.112:0/1120088646 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a75a0f7d0 con 0x556a75ccddc0 2024-04-24T23:17:05.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.396+0000 7f3182ffd640 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccddc0 0x556a75cf50f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3174009850 tx=0x7f3174005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f2c195913cd479 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.396+0000 7f3166ff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-24T23:17:05.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.397+0000 7f3166ff5640 1 -- 172.21.15.112:0/1120088646 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174035040 con 0x556a75ccddc0 2024-04-24T23:17:05.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.397+0000 7f318c5b0740 1 -- 172.21.15.112:0/1120088646 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31780183f0 con 0x556a75ccddc0 2024-04-24T23:17:05.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.397+0000 7f3166ff5640 1 -- 172.21.15.112:0/1120088646 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f3174044dc0 con 0x556a75ccddc0 2024-04-24T23:17:05.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.397+0000 7f3166ff5640 1 -- 172.21.15.112:0/1120088646 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3174053050 con 0x556a75ccddc0 2024-04-24T23:17:05.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.401+0000 7f318c5b0740 10 client.25537 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.401+0000 7f318c5b0740 1 -- 172.21.15.112:0/1120088646 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556a75cf78c0 con 0x556a75ccddc0 2024-04-24T23:17:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.402+0000 7f3166ff5640 1 -- 172.21.15.112:0/1120088646 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f31780183f0 con 0x556a75ccddc0 2024-04-24T23:17:05.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.402+0000 7f3166ff5640 1 -- 172.21.15.112:0/1120088646 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f3174031740 con 0x556a75ccddc0 2024-04-24T23:17:05.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.403+0000 7f318c5b0740 10 client.25537 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.403+0000 7f318c5b0740 10 client.25537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:17:05.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.403+0000 7f318c5b0740 20 client.25537 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.403+0000 7f318c5b0740 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x556a75d4a5b0 0x556a75d4c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.403+0000 7f318c5b0740 4 client.25537 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:05.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.403+0000 7f318c5b0740 1 -- 172.21.15.112:0/1120088646 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a75c38f00 con 0x556a75d4a5b0 2024-04-24T23:17:05.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.406+0000 7f31827fc640 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x556a75d4a5b0 0x556a75d4c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.408+0000 7f31827fc640 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x556a75d4a5b0 0x556a75d4c9d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=438e15fcbc53dc61 server_cookie=d6dde01925d340c3 in_seq=0 out_seq=0 2024-04-24T23:17:05.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.408+0000 7f3166ff5640 10 client.25537 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:17:05.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.415+0000 7f3166ff5640 1 -- 172.21.15.112:0/1120088646 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a75c38f00 con 0x556a75d4a5b0 2024-04-24T23:17:05.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.415+0000 7f3166ff5640 10 client.25537 handle_command_reply: tid=0 2024-04-24T23:17:05.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.415+0000 7f318c5b0740 1 client.25537 shutdown 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.421+0000 7f318c5b0740 1 -- 172.21.15.112:0/1120088646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccddc0 msgr2=0x556a75cf50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.421+0000 7f318c5b0740 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccddc0 0x556a75cf50f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3174009850 tx=0x7f3174005b50 comp rx=0 tx=0).stop 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.422+0000 7f318c5b0740 1 -- 172.21.15.112:0/1120088646 shutdown_connections 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.422+0000 7f318c5b0740 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x556a75d4a5b0 0x556a75d4c9d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.422+0000 7f318c5b0740 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556a75cd2f30 0x556a75cf6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.422+0000 7f318c5b0740 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a75ccddc0 0x556a75cf50f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.422+0000 7f318c5b0740 1 --2- 172.21.15.112:0/1120088646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x556a75cc7330 0x556a75cf4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.422+0000 7f318c5b0740 1 -- 172.21.15.112:0/1120088646 >> 172.21.15.112:0/1120088646 conn(0x556a75cc8a40 msgr2=0x556a75cc7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.423+0000 7f3166ff5640 0 client.25537 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:17:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.425+0000 7f318c5b0740 1 -- 172.21.15.112:0/1120088646 shutdown_connections 2024-04-24T23:17:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.425+0000 7f318c5b0740 1 -- 172.21.15.112:0/1120088646 wait complete. 2024-04-24T23:17:05.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.425+0000 7f318c5b0740 20 client.25537 trim_cache size 0 max 16384 2024-04-24T23:17:05.428 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:05.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.427+0000 7f31897ce640 1 -- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f316c0a2d90 msgr2=0x7f316c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.427+0000 7f31897ce640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f316c0a2d90 0x7f316c0a5250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f317c0a3fa0 tx=0x7f317c005ae0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.427+0000 7f31897ce640 1 -- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184109820 msgr2=0x7f31841adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.427+0000 7f31897ce640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184109820 0x7f31841adea0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f317c00c9e0 tx=0x7f317c00cea0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.427+0000 7f31897ce640 1 -- 172.21.15.112:0/826576391 shutdown_connections 2024-04-24T23:17:05.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.427+0000 7f31897ce640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f316c0a2d90 0x7f316c0a5250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.428+0000 7f31897ce640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3184109820 0x7f31841adea0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.428+0000 7f31897ce640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3184108e80 0x7f31841a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.428+0000 7f31897ce640 1 --2- 172.21.15.112:0/826576391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3184072c30 0x7f31841a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.428+0000 7f31897ce640 1 -- 172.21.15.112:0/826576391 >> 172.21.15.112:0/826576391 conn(0x7f318406e650 msgr2=0x7f31840701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.428+0000 7f31897ce640 1 -- 172.21.15.112:0/826576391 shutdown_connections 2024-04-24T23:17:05.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.428+0000 7f31897ce640 1 -- 172.21.15.112:0/826576391 wait complete. 2024-04-24T23:17:05.450 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:17:05.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.638+0000 7f49fedee640 1 -- 172.21.15.112:0/2576593529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f810dd40 msgr2=0x7f49f81145d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.638+0000 7f49fedee640 1 --2- 172.21.15.112:0/2576593529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f810dd40 0x7f49f81145d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009900 tx=0x7f49e802a400 comp rx=0 tx=0).stop 2024-04-24T23:17:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.641+0000 7f49fedee640 1 -- 172.21.15.112:0/2576593529 shutdown_connections 2024-04-24T23:17:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.641+0000 7f49fedee640 1 --2- 172.21.15.112:0/2576593529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f810dd40 0x7f49f81145d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.641+0000 7f49fedee640 1 --2- 172.21.15.112:0/2576593529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f810d380 0x7f49f810d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.641+0000 7f49fedee640 1 --2- 172.21.15.112:0/2576593529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f810c190 0x7f49f810c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.641+0000 7f49fedee640 1 -- 172.21.15.112:0/2576593529 >> 172.21.15.112:0/2576593529 conn(0x7f49f806c170 msgr2=0x7f49f806c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.642+0000 7f49fedee640 1 -- 172.21.15.112:0/2576593529 shutdown_connections 2024-04-24T23:17:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.642+0000 7f49fedee640 1 -- 172.21.15.112:0/2576593529 wait complete. 2024-04-24T23:17:05.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.642+0000 7f49fedee640 1 Processor -- start 2024-04-24T23:17:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.643+0000 7f49fedee640 1 -- start start 2024-04-24T23:17:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49fedee640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f810c190 0x7f49f81a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49fedee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f810d380 0x7f49f81a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49fedee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f810dd40 0x7f49f81ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49fedee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8117f80 con 0x7f49f810dd40 2024-04-24T23:17:05.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49fedee640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8117e00 con 0x7f49f810c190 2024-04-24T23:17:05.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49fedee640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8118100 con 0x7f49f810d380 2024-04-24T23:17:05.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f810d380 0x7f49f81a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f810d380 0x7f49f81a6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36866/0 (socket says 172.21.15.112:36866) 2024-04-24T23:17:05.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49f7fff640 1 -- 172.21.15.112:0/2912944398 learned_addr learned my addr 172.21.15.112:0/2912944398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.644+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f810c190 0x7f49f81a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.645+0000 7f49fcb8a640 1 -- 172.21.15.112:0/2912944398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f810d380 msgr2=0x7f49f81a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.645+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f810d380 0x7f49f81a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.645+0000 7f49fcb8a640 1 -- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f810dd40 msgr2=0x7f49f81ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.645+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f810dd40 0x7f49f81ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.645+0000 7f49fcb8a640 1 -- 172.21.15.112:0/2912944398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f81ae4e0 con 0x7f49f810c190 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.645+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f810c190 0x7f49f81a68c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005fd0 tx=0x7f49f000ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.646+0000 7f49f5ffb640 1 -- 172.21.15.112:0/2912944398 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f000cde0 con 0x7f49f810c190 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.646+0000 7f49f5ffb640 1 -- 172.21.15.112:0/2912944398 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f49f0011970 con 0x7f49f810c190 2024-04-24T23:17:05.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.646+0000 7f49f5ffb640 1 -- 172.21.15.112:0/2912944398 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f001ab00 con 0x7f49f810c190 2024-04-24T23:17:05.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.647+0000 7f49fedee640 1 -- 172.21.15.112:0/2912944398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f81ae7d0 con 0x7f49f810c190 2024-04-24T23:17:05.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.647+0000 7f49fedee640 1 -- 172.21.15.112:0/2912944398 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81aecd0 con 0x7f49f810c190 2024-04-24T23:17:05.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.652+0000 7f4a01bd8740 1 Processor -- start 2024-04-24T23:17:05.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.652+0000 7f49f5ffb640 1 -- 172.21.15.112:0/2912944398 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f49f001aca0 con 0x7f49f810c190 2024-04-24T23:17:05.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.653+0000 7f49f5ffb640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f49e00a2da0 0x7f49e00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.653+0000 7f4a01bd8740 1 -- start start 2024-04-24T23:17:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f49f5ffb640 1 -- 172.21.15.112:0/2912944398 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f49f00cfe80 con 0x7f49f810c190 2024-04-24T23:17:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 0x55a25fd6e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6ec20 0x55a25fd73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd76220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a25fcd2670 con 0x55a25fd73d90 2024-04-24T23:17:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a25fc7a970 con 0x55a25fd6ec20 2024-04-24T23:17:05.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a25fcce630 con 0x55a25fd6e230 2024-04-24T23:17:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6ec20 0x55a25fd73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6ec20 0x55a25fd73720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50284/0 (socket says 172.21.15.112:50284) 2024-04-24T23:17:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.654+0000 7f49f7fff640 1 -- 172.21.15.112:0/2184063880 learned_addr learned my addr 172.21.15.112:0/2184063880 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.655+0000 7f49fd38b640 1 --2- 172.21.15.112:0/2184063880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd76220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.655+0000 7f49f7fff640 1 -- 172.21.15.112:0/2184063880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 msgr2=0x55a25fd6e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.655+0000 7f49f7fff640 1 --2- 172.21.15.112:0/2184063880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 0x55a25fd6e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.655+0000 7f49f7fff640 1 -- 172.21.15.112:0/2184063880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 msgr2=0x55a25fd76220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.655+0000 7f49f7fff640 1 --2- 172.21.15.112:0/2184063880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd76220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.655+0000 7f49f7fff640 1 -- 172.21.15.112:0/2184063880 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a25fc9e400 con 0x55a25fd6ec20 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.655+0000 7f49f7fff640 1 --2- 172.21.15.112:0/2184063880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6ec20 0x55a25fd73720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00ac20 tx=0x7f49ec00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac50b7699d32bef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.656+0000 7f49deffd640 1 -- 172.21.15.112:0/2184063880 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec01d070 con 0x55a25fd6ec20 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.656+0000 7f49deffd640 1 -- 172.21.15.112:0/2184063880 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f49ec018b40 con 0x55a25fd6ec20 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.656+0000 7f49deffd640 1 -- 172.21.15.112:0/2184063880 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec010ba0 con 0x55a25fd6ec20 2024-04-24T23:17:05.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.656+0000 7f49fd38b640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f49e00a2da0 0x7f49e00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.657+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2184063880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6ec20 msgr2=0x55a25fd73720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.657+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2184063880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6ec20 0x55a25fd73720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00ac20 tx=0x7f49ec00f790 comp rx=0 tx=0).stop 2024-04-24T23:17:05.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.658+0000 7f49fd38b640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f49e00a2da0 0x7f49e00a5260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f49e80096c0 tx=0x7f49e8031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.660+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2184063880 shutdown_connections 2024-04-24T23:17:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.660+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2184063880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd76220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.660+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2184063880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6ec20 0x55a25fd73720 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.660+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2184063880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 0x55a25fd6e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.660+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2184063880 >> 172.21.15.112:0/2184063880 conn(0x55a25fd698a0 msgr2=0x55a25fd69ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.663+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2184063880 shutdown_connections 2024-04-24T23:17:05.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.667+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2184063880 wait complete. 2024-04-24T23:17:05.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 1 -- start start 2024-04-24T23:17:05.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 1 Processor -- start 2024-04-24T23:17:05.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6e230 0x55a25fd78470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd73d90 0x55a25fd7f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a25fd76d40 con 0x55a25fd6e230 2024-04-24T23:17:05.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55a25fc8e050 con 0x55a25fd73d90 2024-04-24T23:17:05.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a25fd76ec0 con 0x55a25fd6ec20 2024-04-24T23:17:05.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.668+0000 7f4a01bd8740 10 client.0 _ll_register_callbacks cb 0x55a25f9e5be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.669+0000 7f49fcb8a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6e230 0x55a25fd78470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.669+0000 7f49fcb8a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6e230 0x55a25fd78470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48466/0 (socket says 172.21.15.112:48466) 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.669+0000 7f49fcb8a640 1 -- 172.21.15.112:0/924010349 learned_addr learned my addr 172.21.15.112:0/924010349 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.669+0000 7f49f7fff640 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.670+0000 7f49fd38b640 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd73d90 0x55a25fd7f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.670+0000 7f49fcb8a640 1 -- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 msgr2=0x55a25fd789b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.670+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd789b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.670+0000 7f49fcb8a640 1 -- 172.21.15.112:0/924010349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd73d90 msgr2=0x55a25fd7f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.670+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd73d90 0x55a25fd7f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.670+0000 7f49fcb8a640 1 -- 172.21.15.112:0/924010349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a25fc5fdc0 con 0x55a25fd6e230 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.670+0000 7f49f7fff640 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd789b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:05.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.672+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6e230 0x55a25fd78470 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f49f0098ce0 tx=0x7f49f009bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc44ed345187694 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.672+0000 7f49d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:05.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.672+0000 7f49d7fff640 1 -- 172.21.15.112:0/924010349 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f00b5070 con 0x55a25fd6e230 2024-04-24T23:17:05.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.672+0000 7f49d7fff640 1 -- 172.21.15.112:0/924010349 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f49f00a4af0 con 0x55a25fd6e230 2024-04-24T23:17:05.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.672+0000 7f4a01bd8740 1 -- 172.21.15.112:0/924010349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49ec018b40 con 0x55a25fd6e230 2024-04-24T23:17:05.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.673+0000 7f49d7fff640 1 -- 172.21.15.112:0/924010349 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f49f00a06c0 con 0x55a25fd6e230 2024-04-24T23:17:05.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.673+0000 7f4a01bd8740 10 client.16230 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.673+0000 7f4a01bd8740 1 -- 172.21.15.112:0/924010349 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a25f9e63f0 con 0x55a25fd6e230 2024-04-24T23:17:05.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.674+0000 7f49d7fff640 1 -- 172.21.15.112:0/924010349 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f49ec018b40 con 0x55a25fd6e230 2024-04-24T23:17:05.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.675+0000 7f49d7fff640 1 -- 172.21.15.112:0/924010349 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f49f0022070 con 0x55a25fd6e230 2024-04-24T23:17:05.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.675+0000 7f4a01bd8740 10 client.16230 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.675+0000 7f4a01bd8740 10 client.16230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:17:05.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.675+0000 7f4a01bd8740 20 client.16230 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.675+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a25fd84760 0x55a25fd86b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.675+0000 7f4a01bd8740 4 client.16230 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:05.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.675+0000 7f4a01bd8740 1 -- 172.21.15.112:0/924010349 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a25fcdfbb0 con 0x55a25fd84760 2024-04-24T23:17:05.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.675+0000 7f49f7fff640 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a25fd84760 0x55a25fd86b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.678+0000 7f49f7fff640 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a25fd84760 0x55a25fd86b90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d235b79c408a147 server_cookie=d62d6f8ea93bd885 in_seq=0 out_seq=0 2024-04-24T23:17:05.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.678+0000 7f49d7fff640 10 client.16230 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:17:05.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.679+0000 7f49d7fff640 1 -- 172.21.15.112:0/924010349 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a25fcdfbb0 con 0x55a25fd84760 2024-04-24T23:17:05.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.680+0000 7f49d7fff640 10 client.16230 handle_command_reply: tid=0 2024-04-24T23:17:05.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.680+0000 7f4a01bd8740 1 client.16230 shutdown 2024-04-24T23:17:05.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.681+0000 7f4a01bd8740 1 -- 172.21.15.112:0/924010349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6e230 msgr2=0x55a25fd78470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.681+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6e230 0x55a25fd78470 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f49f0098ce0 tx=0x7f49f009bf40 comp rx=0 tx=0).stop 2024-04-24T23:17:05.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.683+0000 7f4a01bd8740 1 -- 172.21.15.112:0/924010349 shutdown_connections 2024-04-24T23:17:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.683+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a25fd84760 0x55a25fd86b90 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.683+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd73d90 0x55a25fd7f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.683+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd789b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.683+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/924010349 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6e230 0x55a25fd78470 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.683+0000 7f4a01bd8740 1 -- 172.21.15.112:0/924010349 >> 172.21.15.112:0/924010349 conn(0x55a25fd698a0 msgr2=0x55a25fd68b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.683+0000 7f49d7fff640 0 client.16230 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:17:05.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.683+0000 7f4a01bd8740 1 -- 172.21.15.112:0/924010349 shutdown_connections 2024-04-24T23:17:05.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.684+0000 7f4a01bd8740 1 -- 172.21.15.112:0/924010349 wait complete. 2024-04-24T23:17:05.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.687+0000 7f4a01bd8740 20 client.16230 trim_cache size 0 max 16384 2024-04-24T23:17:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.719+0000 7f4a01bd8740 1 Processor -- start 2024-04-24T23:17:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.719+0000 7f4a01bd8740 1 -- start start 2024-04-24T23:17:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.719+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6e230 0x55a25fd78910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.719+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd7f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.719+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd6cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.719+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81c1b80 con 0x55a25fd73d90 2024-04-24T23:17:05.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.719+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81c37e0 con 0x55a25fd6e230 2024-04-24T23:17:05.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.719+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81c2250 con 0x55a25fd6ec20 2024-04-24T23:17:05.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.720+0000 7f49fd38b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd6cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.720+0000 7f49fd38b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd6cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48474/0 (socket says 172.21.15.112:48474) 2024-04-24T23:17:05.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.721+0000 7f49fd38b640 1 -- 172.21.15.112:0/1384742372 learned_addr learned my addr 172.21.15.112:0/1384742372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.721+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6e230 0x55a25fd78910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.721+0000 7f49f7fff640 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd7f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.723+0000 7f49fd38b640 1 -- 172.21.15.112:0/1384742372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 msgr2=0x55a25fd7f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.723+0000 7f49fd38b640 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd7f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.723+0000 7f49fd38b640 1 -- 172.21.15.112:0/1384742372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6e230 msgr2=0x55a25fd78910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.723+0000 7f49fd38b640 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6e230 0x55a25fd78910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.723+0000 7f49fd38b640 1 -- 172.21.15.112:0/1384742372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a25fab07d0 con 0x55a25fd73d90 2024-04-24T23:17:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.724+0000 7f49fd38b640 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd6cec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005370 tx=0x7f49e803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b73e546de10d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.724+0000 7f49de7fc640 1 -- 172.21.15.112:0/1384742372 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e804d070 con 0x55a25fd73d90 2024-04-24T23:17:05.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.724+0000 7f49de7fc640 1 -- 172.21.15.112:0/1384742372 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f49e8004c40 con 0x55a25fd73d90 2024-04-24T23:17:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.726+0000 7f4a01bd8740 1 -- 172.21.15.112:0/1384742372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 msgr2=0x55a25fd6cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.726+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd6cec0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005370 tx=0x7f49e803c000 comp rx=0 tx=0).stop 2024-04-24T23:17:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.726+0000 7f4a01bd8740 1 -- 172.21.15.112:0/1384742372 shutdown_connections 2024-04-24T23:17:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.726+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd73d90 0x55a25fd6cec0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.726+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6ec20 0x55a25fd7f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.726+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/1384742372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd6e230 0x55a25fd78910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.726+0000 7f4a01bd8740 1 -- 172.21.15.112:0/1384742372 >> 172.21.15.112:0/1384742372 conn(0x55a25fd698a0 msgr2=0x55a25fd6c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.728+0000 7f4a01bd8740 1 -- 172.21.15.112:0/1384742372 shutdown_connections 2024-04-24T23:17:05.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.728+0000 7f4a01bd8740 1 -- 172.21.15.112:0/1384742372 wait complete. 2024-04-24T23:17:05.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 1 -- start start 2024-04-24T23:17:05.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 1 Processor -- start 2024-04-24T23:17:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 0x55a25fd7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6ec20 0x55a25fd7cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd7fae0 0x55a25fde8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81c3050 con 0x55a25fd6ec20 2024-04-24T23:17:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81c2950 con 0x55a25fd7fae0 2024-04-24T23:17:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81a5a30 con 0x55a25fd6e230 2024-04-24T23:17:05.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.729+0000 7f4a01bd8740 10 client.0 _ll_register_callbacks cb 0x7f49f001a4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.730+0000 7f49fd38b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd7fae0 0x55a25fde8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.730+0000 7f49fd38b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd7fae0 0x55a25fde8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50310/0 (socket says 172.21.15.112:50310) 2024-04-24T23:17:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.730+0000 7f49fd38b640 1 -- 172.21.15.112:0/2431037953 learned_addr learned my addr 172.21.15.112:0/2431037953 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.730+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 0x55a25fd7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.730+0000 7f49f7fff640 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6ec20 0x55a25fd7cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.732+0000 7f49f7fff640 1 -- 172.21.15.112:0/2431037953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 msgr2=0x55a25fd7c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.732+0000 7f49f7fff640 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 0x55a25fd7c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.732+0000 7f49f7fff640 1 -- 172.21.15.112:0/2431037953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd7fae0 msgr2=0x55a25fde8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.732+0000 7f49f7fff640 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd7fae0 0x55a25fde8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.732+0000 7f49f7fff640 1 -- 172.21.15.112:0/2431037953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a25fde9270 con 0x55a25fd6ec20 2024-04-24T23:17:05.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.733+0000 7f49f7fff640 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6ec20 0x55a25fd7cac0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55a25fd6a770 tx=0x7f49ec005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca53acfd4db8d828 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.733+0000 7f49d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.736+0000 7f49d7fff640 1 -- 172.21.15.112:0/2431037953 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec01d040 con 0x55a25fd6ec20 2024-04-24T23:17:05.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.736+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2431037953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a25fcd9f00 con 0x55a25fd6ec20 2024-04-24T23:17:05.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.736+0000 7f49d7fff640 1 -- 172.21.15.112:0/2431037953 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f49ec005380 con 0x55a25fd6ec20 2024-04-24T23:17:05.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.739+0000 7f49d7fff640 1 -- 172.21.15.112:0/2431037953 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f49ec005580 con 0x55a25fd6ec20 2024-04-24T23:17:05.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.740+0000 7f4a01bd8740 10 client.16236 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.740+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2431037953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a25fdea600 con 0x55a25fd6ec20 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f49d7fff640 1 -- 172.21.15.112:0/2431037953 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x55a25fcd9f00 con 0x55a25fd6ec20 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f49d7fff640 1 -- 172.21.15.112:0/2431037953 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f49ec004ba0 con 0x55a25fd6ec20 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f4a01bd8740 10 client.16236 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f4a01bd8740 10 client.16236 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f4a01bd8740 20 client.16236 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a25fdeb010 0x55a25fded430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f4a01bd8740 4 client.16236 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2431037953 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a25fc57820 con 0x55a25fdeb010 2024-04-24T23:17:05.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.742+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a25fdeb010 0x55a25fded430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.744+0000 7f49fcb8a640 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a25fdeb010 0x55a25fded430 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef247ba0cbd2e6fa server_cookie=781c6e4e44abd1a4 in_seq=0 out_seq=0 2024-04-24T23:17:05.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.744+0000 7f49d7fff640 10 client.16236 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:17:05.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.744+0000 7f49d7fff640 1 -- 172.21.15.112:0/2431037953 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a25fc57820 con 0x55a25fdeb010 2024-04-24T23:17:05.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.744+0000 7f49d7fff640 10 client.16236 handle_command_reply: tid=0 2024-04-24T23:17:05.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.745+0000 7f4a01bd8740 1 client.16236 shutdown 2024-04-24T23:17:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.745+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2431037953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6ec20 msgr2=0x55a25fd7cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.745+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6ec20 0x55a25fd7cac0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55a25fd6a770 tx=0x7f49ec005c90 comp rx=0 tx=0).stop 2024-04-24T23:17:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.746+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2431037953 shutdown_connections 2024-04-24T23:17:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.746+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55a25fdeb010 0x55a25fded430 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.746+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55a25fd7fae0 0x55a25fde8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.746+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a25fd6ec20 0x55a25fd7cac0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.746+0000 7f4a01bd8740 1 --2- 172.21.15.112:0/2431037953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a25fd6e230 0x55a25fd7c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.746+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2431037953 >> 172.21.15.112:0/2431037953 conn(0x55a25fd698a0 msgr2=0x55a25fd6d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.746+0000 7f49d7fff640 0 client.16236 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:17:05.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.746+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2431037953 shutdown_connections 2024-04-24T23:17:05.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.748+0000 7f4a01bd8740 1 -- 172.21.15.112:0/2431037953 wait complete. 2024-04-24T23:17:05.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.750+0000 7f4a01bd8740 20 client.16236 trim_cache size 0 max 16384 2024-04-24T23:17:05.755 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:05.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.754+0000 7f49fedee640 1 -- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f49e00a2da0 msgr2=0x7f49e00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.754+0000 7f49fedee640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f49e00a2da0 0x7f49e00a5260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f49e80096c0 tx=0x7f49e8031640 comp rx=0 tx=0).stop 2024-04-24T23:17:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.755+0000 7f49fedee640 1 -- 172.21.15.112:0/2912944398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f810c190 msgr2=0x7f49f81a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.755+0000 7f49fedee640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f810c190 0x7f49f81a68c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49f0005fd0 tx=0x7f49f000ef30 comp rx=0 tx=0).stop 2024-04-24T23:17:05.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.755+0000 7f49fedee640 1 -- 172.21.15.112:0/2912944398 shutdown_connections 2024-04-24T23:17:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.755+0000 7f49fedee640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f49e00a2da0 0x7f49e00a5260 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.755+0000 7f49fedee640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49f810dd40 0x7f49f81ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.756+0000 7f49fedee640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49f810d380 0x7f49f81a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.756+0000 7f49fedee640 1 --2- 172.21.15.112:0/2912944398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49f810c190 0x7f49f81a68c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.756+0000 7f49fedee640 1 -- 172.21.15.112:0/2912944398 >> 172.21.15.112:0/2912944398 conn(0x7f49f806c170 msgr2=0x7f49f806f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.758+0000 7f49fedee640 1 -- 172.21.15.112:0/2912944398 shutdown_connections 2024-04-24T23:17:05.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.758+0000 7f49fedee640 1 -- 172.21.15.112:0/2912944398 wait complete. 2024-04-24T23:17:05.783 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:17:05.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.958+0000 7ff873fa5640 1 -- 172.21.15.112:0/870188157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 msgr2=0x7ff86c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.958+0000 7ff873fa5640 1 --2- 172.21.15.112:0/870188157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 0x7ff86c100aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff864009ca0 tx=0x7ff86402a8e0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.958+0000 7ff873fa5640 1 -- 172.21.15.112:0/870188157 shutdown_connections 2024-04-24T23:17:05.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.958+0000 7ff873fa5640 1 --2- 172.21.15.112:0/870188157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff86c102230 0x7ff86c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.958+0000 7ff873fa5640 1 --2- 172.21.15.112:0/870188157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86c101870 0x7ff86c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.958+0000 7ff873fa5640 1 --2- 172.21.15.112:0/870188157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 0x7ff86c100aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.958+0000 7ff873fa5640 1 -- 172.21.15.112:0/870188157 >> 172.21.15.112:0/870188157 conn(0x7ff86c0fbbf0 msgr2=0x7ff86c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.960+0000 7ff873fa5640 1 -- 172.21.15.112:0/870188157 shutdown_connections 2024-04-24T23:17:05.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.960+0000 7ff873fa5640 1 -- 172.21.15.112:0/870188157 wait complete. 2024-04-24T23:17:05.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.961+0000 7ff873fa5640 1 Processor -- start 2024-04-24T23:17:05.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.961+0000 7ff873fa5640 1 -- start start 2024-04-24T23:17:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.961+0000 7ff873fa5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 0x7ff86c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.961+0000 7ff873fa5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff86c101870 0x7ff86c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.961+0000 7ff873fa5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86c102230 0x7ff86c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.961+0000 7ff873fa5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c10c500 con 0x7ff86c102230 2024-04-24T23:17:05.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.961+0000 7ff873fa5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c10c380 con 0x7ff86c100680 2024-04-24T23:17:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.961+0000 7ff873fa5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c10c680 con 0x7ff86c101870 2024-04-24T23:17:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.962+0000 7ff872542640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86c102230 0x7ff86c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.962+0000 7ff872542640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86c102230 0x7ff86c1a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48490/0 (socket says 172.21.15.112:48490) 2024-04-24T23:17:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.962+0000 7ff872542640 1 -- 172.21.15.112:0/872323603 learned_addr learned my addr 172.21.15.112:0/872323603 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.962+0000 7ff871540640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff86c101870 0x7ff86c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.962+0000 7ff871d41640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 0x7ff86c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.964+0000 7ff871d41640 1 -- 172.21.15.112:0/872323603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff86c101870 msgr2=0x7ff86c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.964+0000 7ff871d41640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff86c101870 0x7ff86c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.964+0000 7ff871d41640 1 -- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86c102230 msgr2=0x7ff86c1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.964+0000 7ff871d41640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86c102230 0x7ff86c1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.964+0000 7ff871d41640 1 -- 172.21.15.112:0/872323603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86c198780 con 0x7ff86c100680 2024-04-24T23:17:05.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.964+0000 7ff871d41640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 0x7ff86c069440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff864009bf0 tx=0x7ff864003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.965+0000 7ff862ffd640 1 -- 172.21.15.112:0/872323603 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864004040 con 0x7ff86c100680 2024-04-24T23:17:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.965+0000 7ff862ffd640 1 -- 172.21.15.112:0/872323603 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff8640046e0 con 0x7ff86c100680 2024-04-24T23:17:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.965+0000 7ff862ffd640 1 -- 172.21.15.112:0/872323603 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86402f490 con 0x7ff86c100680 2024-04-24T23:17:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.965+0000 7ff873fa5640 1 -- 172.21.15.112:0/872323603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86c1989b0 con 0x7ff86c100680 2024-04-24T23:17:05.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.965+0000 7ff873fa5640 1 -- 172.21.15.112:0/872323603 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86c198f00 con 0x7ff86c100680 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.968+0000 7ff876d8c740 1 Processor -- start 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff876d8c740 1 -- start start 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84a3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84a8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a8d90 0x555ad84ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff876d8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ad8407670 con 0x555ad84a3c20 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff876d8c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555ad83af970 con 0x555ad84a3230 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff876d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555ad8403630 con 0x555ad84a8d90 2024-04-24T23:17:05.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff871540640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a8d90 0x555ad84ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff871540640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a8d90 0x555ad84ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36920/0 (socket says 172.21.15.112:36920) 2024-04-24T23:17:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.969+0000 7ff871540640 1 -- 172.21.15.112:0/1424676182 learned_addr learned my addr 172.21.15.112:0/1424676182 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.970+0000 7ff872542640 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.970+0000 7ff871540640 1 -- 172.21.15.112:0/1424676182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 msgr2=0x555ad84a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.971+0000 7ff871540640 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.971+0000 7ff871540640 1 -- 172.21.15.112:0/1424676182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 msgr2=0x555ad84a8720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-24T23:17:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.971+0000 7ff871540640 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84a8720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.971+0000 7ff871540640 1 -- 172.21.15.112:0/1424676182 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad83d3400 con 0x555ad84a8d90 2024-04-24T23:17:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.971+0000 7ff872542640 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.971+0000 7ff862ffd640 1 -- 172.21.15.112:0/872323603 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff8640305e0 con 0x7ff86c100680 2024-04-24T23:17:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.971+0000 7ff862ffd640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8580a2cc0 0x7ff8580a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.972+0000 7ff862ffd640 1 -- 172.21.15.112:0/872323603 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff8640f0a80 con 0x7ff86c100680 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.973+0000 7ff871540640 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a8d90 0x555ad84ab220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009a10 tx=0x7ff85c009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6551c86425fa75 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.973+0000 7ff872542640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8580a2cc0 0x7ff8580a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.973+0000 7ff857fff640 1 -- 172.21.15.112:0/1424676182 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c017070 con 0x555ad84a8d90 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.973+0000 7ff857fff640 1 -- 172.21.15.112:0/1424676182 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff85c013470 con 0x555ad84a8d90 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.973+0000 7ff857fff640 1 -- 172.21.15.112:0/1424676182 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c012610 con 0x555ad84a8d90 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.974+0000 7ff876d8c740 1 -- 172.21.15.112:0/1424676182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a8d90 msgr2=0x555ad84ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.974+0000 7ff876d8c740 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a8d90 0x555ad84ab220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009a10 tx=0x7ff85c009ed0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.974+0000 7ff872542640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8580a2cc0 0x7ff8580a5180 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x555ad84a4480 tx=0x7ff868005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.975+0000 7ff876d8c740 1 -- 172.21.15.112:0/1424676182 shutdown_connections 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.975+0000 7ff876d8c740 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a8d90 0x555ad84ab220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.975+0000 7ff876d8c740 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.975+0000 7ff876d8c740 1 --2- 172.21.15.112:0/1424676182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.975+0000 7ff876d8c740 1 -- 172.21.15.112:0/1424676182 >> 172.21.15.112:0/1424676182 conn(0x555ad849e8a0 msgr2=0x555ad849eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.975+0000 7ff876d8c740 1 -- 172.21.15.112:0/1424676182 shutdown_connections 2024-04-24T23:17:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.975+0000 7ff876d8c740 1 -- 172.21.15.112:0/1424676182 wait complete. 2024-04-24T23:17:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.976+0000 7ff876d8c740 1 -- start start 2024-04-24T23:17:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.976+0000 7ff876d8c740 1 Processor -- start 2024-04-24T23:17:05.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.976+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3c20 0x555ad84be920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff876d8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555ad84bcdf0 con 0x555ad84a8d90 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff876d8c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x555ad83c3050 con 0x555ad84a3c20 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff876d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555ad84bcf70 con 0x555ad84a3230 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff876d8c740 10 client.0 _ll_register_callbacks cb 0x555ad811abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff871d41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff871d41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36930/0 (socket says 172.21.15.112:36930) 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff871d41640 1 -- 172.21.15.112:0/3931220503 learned_addr learned my addr 172.21.15.112:0/3931220503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.977+0000 7ff872542640 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff871540640 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3c20 0x555ad84be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff871d41640 1 -- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3c20 msgr2=0x555ad84be920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff871d41640 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3c20 0x555ad84be920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff871d41640 1 -- 172.21.15.112:0/3931220503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 msgr2=0x555ad84c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff871d41640 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff871d41640 1 -- 172.21.15.112:0/3931220503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad8394dc0 con 0x555ad84a3230 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff871540640 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3c20 0x555ad84be920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff871d41640 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be3e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8640bce20 tx=0x7ff8640b9130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2697fbbeedc22d19 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.978+0000 7ff8557fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.979+0000 7ff8557fa640 1 -- 172.21.15.112:0/3931220503 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8640bc7f0 con 0x555ad84a3230 2024-04-24T23:17:05.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.979+0000 7ff876d8c740 1 -- 172.21.15.112:0/3931220503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85c013470 con 0x555ad84a3230 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.979+0000 7ff8557fa640 1 -- 172.21.15.112:0/3931220503 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff864004b40 con 0x555ad84a3230 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.979+0000 7ff8557fa640 1 -- 172.21.15.112:0/3931220503 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff8640bd590 con 0x555ad84a3230 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.979+0000 7ff876d8c740 10 client.25561 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.979+0000 7ff876d8c740 1 -- 172.21.15.112:0/3931220503 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555ad811b3f0 con 0x555ad84a3230 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.979+0000 7ff8557fa640 1 -- 172.21.15.112:0/3931220503 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff85c013470 con 0x555ad84a3230 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.979+0000 7ff8557fa640 1 -- 172.21.15.112:0/3931220503 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7ff8640bc370 con 0x555ad84a3230 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.980+0000 7ff876d8c740 10 client.25561 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.980+0000 7ff876d8c740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.980+0000 7ff876d8c740 20 client.25561 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.980+0000 7ff876d8c740 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x555ad84aeac0 0x555ad84b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.980+0000 7ff876d8c740 4 client.25561 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.980+0000 7ff876d8c740 1 -- 172.21.15.112:0/3931220503 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ad8414bb0 con 0x555ad84aeac0 2024-04-24T23:17:05.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.980+0000 7ff871540640 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x555ad84aeac0 0x555ad84b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.982+0000 7ff871540640 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x555ad84aeac0 0x555ad84b0f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46892bfe152b005f server_cookie=3254664dbcc03379 in_seq=0 out_seq=0 2024-04-24T23:17:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.982+0000 7ff8557fa640 10 client.25561 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.987+0000 7ff8557fa640 1 -- 172.21.15.112:0/3931220503 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ad8414bb0 con 0x555ad84aeac0 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.987+0000 7ff8557fa640 10 client.25561 handle_command_reply: tid=0 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.987+0000 7ff876d8c740 1 client.25561 shutdown 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 -- 172.21.15.112:0/3931220503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 msgr2=0x555ad84be3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be3e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8640bce20 tx=0x7ff8640b9130 comp rx=0 tx=0).stop 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 -- 172.21.15.112:0/3931220503 shutdown_connections 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x555ad84aeac0 0x555ad84b0f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3c20 0x555ad84be920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 --2- 172.21.15.112:0/3931220503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be3e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:05.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 -- 172.21.15.112:0/3931220503 >> 172.21.15.112:0/3931220503 conn(0x555ad849e8a0 msgr2=0x555ad849db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:05.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff8557fa640 0 client.25561 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:05.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 -- 172.21.15.112:0/3931220503 shutdown_connections 2024-04-24T23:17:05.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 1 -- 172.21.15.112:0/3931220503 wait complete. 2024-04-24T23:17:05.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:05.988+0000 7ff876d8c740 20 client.25561 trim_cache size 0 max 16384 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.009+0000 7ff876d8c740 1 Processor -- start 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.009+0000 7ff876d8c740 1 -- start start 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.009+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.009+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a8d90 0x555ad849d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff871540640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff871540640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48506/0 (socket says 172.21.15.112:48506) 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff871d41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff871d41640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:36944/0 (socket says 172.21.15.112:36944) 2024-04-24T23:17:06.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff871d41640 1 -- 172.21.15.112:0/977010053 learned_addr learned my addr 172.21.15.112:0/977010053 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:06.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff876d8c740 1 -- 172.21.15.112:0/977010053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c071ad0 con 0x555ad84a3c20 2024-04-24T23:17:06.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff876d8c740 1 -- 172.21.15.112:0/977010053 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c073730 con 0x555ad84a8d90 2024-04-24T23:17:06.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff876d8c740 1 -- 172.21.15.112:0/977010053 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c0721a0 con 0x555ad84a3230 2024-04-24T23:17:06.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.010+0000 7ff872542640 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a8d90 0x555ad849d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:06.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff871d41640 1 -- 172.21.15.112:0/977010053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a8d90 msgr2=0x555ad849d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:06.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff871d41640 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a8d90 0x555ad849d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff871d41640 1 -- 172.21.15.112:0/977010053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 msgr2=0x555ad84c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff871d41640 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff871d41640 1 -- 172.21.15.112:0/977010053 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad81e57d0 con 0x555ad84a3230 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff871540640 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff871d41640 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8640b8b70 tx=0x7ff86402a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f98b48f668a541a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff8577fe640 1 -- 172.21.15.112:0/977010053 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8640d9030 con 0x555ad84a3230 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff8577fe640 1 -- 172.21.15.112:0/977010053 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff864036030 con 0x555ad84a3230 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.011+0000 7ff8577fe640 1 -- 172.21.15.112:0/977010053 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8640b9500 con 0x555ad84a3230 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 -- 172.21.15.112:0/977010053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 msgr2=0x555ad84be880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8640b8b70 tx=0x7ff86402a8e0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 -- 172.21.15.112:0/977010053 shutdown_connections 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a8d90 0x555ad849d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a3c20 0x555ad84c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 --2- 172.21.15.112:0/977010053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3230 0x555ad84be880 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 -- 172.21.15.112:0/977010053 >> 172.21.15.112:0/977010053 conn(0x555ad849e8a0 msgr2=0x555ad849d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:06.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 -- 172.21.15.112:0/977010053 shutdown_connections 2024-04-24T23:17:06.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.012+0000 7ff876d8c740 1 -- 172.21.15.112:0/977010053 wait complete. 2024-04-24T23:17:06.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff876d8c740 1 -- start start 2024-04-24T23:17:06.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff876d8c740 1 Processor -- start 2024-04-24T23:17:06.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84c3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:06.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3c20 0x555ad84c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:06.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff876d8c740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:06.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff876d8c740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c072fa0 con 0x555ad84a8d90 2024-04-24T23:17:06.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff871d41640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:06.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff871540640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3c20 0x555ad84c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:06.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff871d41640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84c3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50354/0 (socket says 172.21.15.112:50354) 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff871d41640 1 -- 172.21.15.112:0/706088917 learned_addr learned my addr 172.21.15.112:0/706088917 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.013+0000 7ff876d8c740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c0728a0 con 0x555ad84a3230 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff872542640 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c06fa80 con 0x555ad84a3c20 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff876d8c740 10 client.0 _ll_register_callbacks cb 0x7ff864009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff871540640 1 -- 172.21.15.112:0/706088917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 msgr2=0x555ad84c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff871540640 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff871540640 1 -- 172.21.15.112:0/706088917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 msgr2=0x555ad84c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff871540640 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff871540640 1 -- 172.21.15.112:0/706088917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad84c4fc0 con 0x555ad84a3c20 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff872542640 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff871d41640 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff871540640 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3c20 0x555ad84c39c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0099e0 tx=0x7ff85c009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e4b000382d8a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:06.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff8557fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff8557fa640 1 -- 172.21.15.112:0/706088917 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c017040 con 0x555ad84a3c20 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff864036030 con 0x555ad84a3c20 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.014+0000 7ff8557fa640 1 -- 172.21.15.112:0/706088917 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff85c0258e0 con 0x555ad84a3c20 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff8557fa640 1 -- 172.21.15.112:0/706088917 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff85c005990 con 0x555ad84a3c20 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff876d8c740 10 client.25567 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555ad85415b0 con 0x555ad84a3c20 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff8557fa640 1 -- 172.21.15.112:0/706088917 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff85c004520 con 0x555ad84a3c20 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff8557fa640 1 -- 172.21.15.112:0/706088917 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7ff85c013620 con 0x555ad84a3c20 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff876d8c740 10 client.25567 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff876d8c740 10 client.25567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff876d8c740 20 client.25567 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff876d8c740 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x555ad8541d30 0x555ad8544150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff876d8c740 4 client.25567 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff864036030 con 0x555ad8541d30 2024-04-24T23:17:06.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.015+0000 7ff871d41640 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x555ad8541d30 0x555ad8544150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:06.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.016+0000 7ff8557fa640 10 client.25567 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:06.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.016+0000 7ff871d41640 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x555ad8541d30 0x555ad8544150 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cdf11e1dea42d5f server_cookie=fedcfee4c25786f in_seq=0 out_seq=0 2024-04-24T23:17:06.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff8557fa640 1 -- 172.21.15.112:0/706088917 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff864036030 con 0x555ad8541d30 2024-04-24T23:17:06.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff8557fa640 10 client.25567 handle_command_reply: tid=0 2024-04-24T23:17:06.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 client.25567 shutdown 2024-04-24T23:17:06.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3c20 msgr2=0x555ad84c39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:06.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3c20 0x555ad84c39c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0099e0 tx=0x7ff85c009950 comp rx=0 tx=0).stop 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 shutdown_connections 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x555ad8541d30 0x555ad8544150 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555ad84a8d90 0x555ad84c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ad84a3c20 0x555ad84c39c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 --2- 172.21.15.112:0/706088917 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555ad84a3230 0x555ad84c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 >> 172.21.15.112:0/706088917 conn(0x555ad849e8a0 msgr2=0x555ad849db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff8557fa640 0 client.25567 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 shutdown_connections 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 1 -- 172.21.15.112:0/706088917 wait complete. 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.021+0000 7ff876d8c740 20 client.25567 trim_cache size 0 max 16384 2024-04-24T23:17:06.024 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:06.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 -- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8580a2cc0 msgr2=0x7ff8580a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:06.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8580a2cc0 0x7ff8580a5180 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x555ad84a4480 tx=0x7ff868005f90 comp rx=0 tx=0).stop 2024-04-24T23:17:06.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 -- 172.21.15.112:0/872323603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 msgr2=0x7ff86c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:06.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 0x7ff86c069440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff864009bf0 tx=0x7ff864003ba0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 -- 172.21.15.112:0/872323603 shutdown_connections 2024-04-24T23:17:06.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff8580a2cc0 0x7ff8580a5180 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff86c102230 0x7ff86c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff86c101870 0x7ff86c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 --2- 172.21.15.112:0/872323603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff86c100680 0x7ff86c069440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:06.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 -- 172.21.15.112:0/872323603 >> 172.21.15.112:0/872323603 conn(0x7ff86c0fbbf0 msgr2=0x7ff86c104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:06.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 -- 172.21.15.112:0/872323603 shutdown_connections 2024-04-24T23:17:06.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:06.023+0000 7ff873fa5640 1 -- 172.21.15.112:0/872323603 wait complete. 2024-04-24T23:17:06.039 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:05 smithi112 ceph-mon[29186]: pgmap v433: 129 pgs: 129 active+clean; 181 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 351 op/s 2024-04-24T23:17:06.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:05 smithi187 ceph-mon[33292]: pgmap v433: 129 pgs: 129 active+clean; 181 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 351 op/s 2024-04-24T23:17:06.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:05 smithi150 ceph-mon[32247]: pgmap v433: 129 pgs: 129 active+clean; 181 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 351 op/s 2024-04-24T23:17:07.039 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-24T23:17:07.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:17:07.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-24T23:17:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 -- 172.21.15.112:0/2509936403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4100680 msgr2=0x7f15c4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 --2- 172.21.15.112:0/2509936403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4100680 0x7f15c4100aa0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009900 tx=0x7f15b802a410 comp rx=0 tx=0).stop 2024-04-24T23:17:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 -- 172.21.15.112:0/2509936403 shutdown_connections 2024-04-24T23:17:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 --2- 172.21.15.112:0/2509936403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c4102230 0x7f15c4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 --2- 172.21.15.112:0/2509936403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15c4101870 0x7f15c4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 --2- 172.21.15.112:0/2509936403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4100680 0x7f15c4100aa0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 -- 172.21.15.112:0/2509936403 >> 172.21.15.112:0/2509936403 conn(0x7f15c40fbbf0 msgr2=0x7f15c40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 -- 172.21.15.112:0/2509936403 shutdown_connections 2024-04-24T23:17:07.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.144+0000 7f15c9e53640 1 -- 172.21.15.112:0/2509936403 wait complete. 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c9e53640 1 Processor -- start 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c9e53640 1 -- start start 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c9e53640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15c4100680 0x7f15c4196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c9e53640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c4101870 0x7f15c4197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c9e53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4102230 0x7f15c419e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c9e53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15c410c430 con 0x7f15c4102230 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c9e53640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15c410c2b0 con 0x7f15c4101870 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c9e53640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15c410c5b0 con 0x7f15c4100680 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4102230 0x7f15c419e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4102230 0x7f15c419e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48540/0 (socket says 172.21.15.112:48540) 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c3fff640 1 -- 172.21.15.112:0/3816457224 learned_addr learned my addr 172.21.15.112:0/3816457224 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c4101870 0x7f15c4197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c37fe640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15c4100680 0x7f15c4196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c2ffd640 1 -- 172.21.15.112:0/3816457224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15c4100680 msgr2=0x7f15c4196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.145+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15c4100680 0x7f15c4196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c2ffd640 1 -- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4102230 msgr2=0x7f15c419e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4102230 0x7f15c419e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c2ffd640 1 -- 172.21.15.112:0/3816457224 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c419e6f0 con 0x7f15c4101870 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c3fff640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4102230 0x7f15c419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c4101870 0x7f15c4197010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009ef0 tx=0x7f15b400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c0ff9640 1 -- 172.21.15.112:0/3816457224 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b400ccc0 con 0x7f15c4101870 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c0ff9640 1 -- 172.21.15.112:0/3816457224 <== mon.2 v2:172.21.15.150:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15b4005020 con 0x7f15c4101870 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c9e53640 1 -- 172.21.15.112:0/3816457224 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c4068930 con 0x7f15c4101870 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c0ff9640 1 -- 172.21.15.112:0/3816457224 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4004ad0 con 0x7f15c4101870 2024-04-24T23:17:07.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.146+0000 7f15c9e53640 1 -- 172.21.15.112:0/3816457224 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c4068e30 con 0x7f15c4101870 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.148+0000 7f15c0ff9640 1 -- 172.21.15.112:0/3816457224 <== mon.2 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f15b4004cb0 con 0x7f15c4101870 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15ccc33740 1 Processor -- start 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c0ff9640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15ac0a2b90 0x7f15ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15ccc33740 1 -- start start 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c0ff9640 1 -- 172.21.15.112:0/3816457224 <== mon.2 v2:172.21.15.150:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f15b40d1580 con 0x7f15c4101870 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c3fff640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15ac0a2b90 0x7f15ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef11780 0x5587aef16280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef168f0 0x5587aef18d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15ccc33740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587aee742c0 con 0x5587aef10d90 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15ccc33740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587aee1c910 con 0x5587aef11780 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15ccc33740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587aee70280 con 0x5587aef168f0 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef111b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48544/0 (socket says 172.21.15.112:48544) 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c37fe640 1 -- 172.21.15.112:0/1033999906 learned_addr learned my addr 172.21.15.112:0/1033999906 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef11780 0x5587aef16280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c3fff640 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef168f0 0x5587aef18d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c37fe640 1 -- 172.21.15.112:0/1033999906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef168f0 msgr2=0x5587aef18d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c37fe640 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef168f0 0x5587aef18d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c37fe640 1 -- 172.21.15.112:0/1033999906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef11780 msgr2=0x5587aef16280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c37fe640 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef11780 0x5587aef16280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c37fe640 1 -- 172.21.15.112:0/1033999906 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587aee2dfb0 con 0x5587aef10d90 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.149+0000 7f15c3fff640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15ac0a2b90 0x7f15ac0a5050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009fd0 tx=0x7f15bc00fc00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15c37fe640 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef111b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c773b080dac623b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15a9ffb640 1 -- 172.21.15.112:0/1033999906 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8040070 con 0x5587aef10d90 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15a9ffb640 1 -- 172.21.15.112:0/1033999906 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15b8031a00 con 0x5587aef10d90 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15a9ffb640 1 -- 172.21.15.112:0/1033999906 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8035040 con 0x5587aef10d90 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15ccc33740 1 -- 172.21.15.112:0/1033999906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 msgr2=0x5587aef111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef111b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f15b8000c00 tx=0x7f15b8005bd0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15ccc33740 1 -- 172.21.15.112:0/1033999906 shutdown_connections 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef168f0 0x5587aef18d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef11780 0x5587aef16280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1033999906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef111b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15ccc33740 1 -- 172.21.15.112:0/1033999906 >> 172.21.15.112:0/1033999906 conn(0x5587aeed8950 msgr2=0x5587aef0c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.150+0000 7f15ccc33740 1 -- 172.21.15.112:0/1033999906 shutdown_connections 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 -- 172.21.15.112:0/1033999906 wait complete. 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 -- start start 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 Processor -- start 2024-04-24T23:17:07.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef1f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef1fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5587aef1dc50 con 0x5587aef168f0 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x5587aee2fe90 con 0x5587aef10d90 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587aef1ddd0 con 0x5587aef11780 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15ccc33740 10 client.0 _ll_register_callbacks cb 0x5587aee6b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48558/0 (socket says 172.21.15.112:48558) 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15c37fe640 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef1f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15c3fff640 1 -- 172.21.15.112:0/3314905263 learned_addr learned my addr 172.21.15.112:0/3314905263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.151+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef1fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15c3fff640 1 -- 172.21.15.112:0/3314905263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 msgr2=0x5587aef1fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15c3fff640 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef1fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15c3fff640 1 -- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 msgr2=0x5587aef1f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15c3fff640 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef1f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15c3fff640 1 -- 172.21.15.112:0/3314905263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587aec507e0 con 0x5587aef168f0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15c37fe640 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef1f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15c3fff640 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26b00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f15bc011fd0 tx=0x7f15bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74754a912f11c243 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15a2ffd640 1 -- 172.21.15.112:0/3314905263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc017040 con 0x5587aef168f0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15a2ffd640 1 -- 172.21.15.112:0/3314905263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f15bc0283f0 con 0x5587aef168f0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15ccc33740 1 -- 172.21.15.112:0/3314905263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15b8031a00 con 0x5587aef168f0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15a2ffd640 1 -- 172.21.15.112:0/3314905263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f15bc0285f0 con 0x5587aef168f0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15ccc33740 10 client.16260 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15ccc33740 1 -- 172.21.15.112:0/3314905263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5587aecb0f70 con 0x5587aef168f0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.152+0000 7f15a2ffd640 1 -- 172.21.15.112:0/3314905263 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15b8031a00 con 0x5587aef168f0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15a2ffd640 1 -- 172.21.15.112:0/3314905263 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f15bc00e960 con 0x5587aef168f0 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15ccc33740 10 client.16260 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:07.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15ccc33740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15ccc33740 20 client.16260 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15ccc33740 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5587aef2b140 0x5587aef2d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15ccc33740 4 client.16260 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15ccc33740 1 -- 172.21.15.112:0/3314905263 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587aee485a0 con 0x5587aef2b140 2024-04-24T23:17:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15c37fe640 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5587aef2b140 0x5587aef2d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15c37fe640 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5587aef2b140 0x5587aef2d5e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dcff517c889ea3c server_cookie=982c0d078769024d in_seq=0 out_seq=0 2024-04-24T23:17:07.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.153+0000 7f15a2ffd640 10 client.16260 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15a2ffd640 1 -- 172.21.15.112:0/3314905263 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587aee485a0 con 0x5587aef2b140 2024-04-24T23:17:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15a2ffd640 10 client.16260 handle_command_reply: tid=0 2024-04-24T23:17:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 client.16260 shutdown 2024-04-24T23:17:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 -- 172.21.15.112:0/3314905263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 msgr2=0x5587aef26b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26b00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f15bc011fd0 tx=0x7f15bc004ab0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 -- 172.21.15.112:0/3314905263 shutdown_connections 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5587aef2b140 0x5587aef2d5e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26b00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef1fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 --2- 172.21.15.112:0/3314905263 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef1f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.155+0000 7f15ccc33740 1 -- 172.21.15.112:0/3314905263 >> 172.21.15.112:0/3314905263 conn(0x5587aeed8950 msgr2=0x5587aef0b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.156+0000 7f15a2ffd640 0 client.16260 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.156+0000 7f15ccc33740 1 -- 172.21.15.112:0/3314905263 shutdown_connections 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.156+0000 7f15ccc33740 1 -- 172.21.15.112:0/3314905263 wait complete. 2024-04-24T23:17:07.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.156+0000 7f15ccc33740 20 client.16260 trim_cache size 0 max 16384 2024-04-24T23:17:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15ccc33740 1 Processor -- start 2024-04-24T23:17:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15ccc33740 1 -- start start 2024-04-24T23:17:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef12bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef13130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15ccc33740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41a8420 con 0x5587aef168f0 2024-04-24T23:17:07.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15ccc33740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41aa080 con 0x5587aef10d90 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15ccc33740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41a8af0 con 0x5587aef11780 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.173+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48564/0 (socket says 172.21.15.112:48564) 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c3fff640 1 -- 172.21.15.112:0/2063831697 learned_addr learned my addr 172.21.15.112:0/2063831697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c37fe640 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef12bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef13130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c3fff640 1 -- 172.21.15.112:0/2063831697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 msgr2=0x5587aef13130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c3fff640 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef13130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c3fff640 1 -- 172.21.15.112:0/2063831697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 msgr2=0x5587aef12bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c3fff640 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef12bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c3fff640 1 -- 172.21.15.112:0/2063831697 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587aeb48170 con 0x5587aef168f0 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15c3fff640 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26ac0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00fea0 tx=0x7f15bc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661c307a70f663e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15a97fa640 1 -- 172.21.15.112:0/2063831697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc017050 con 0x5587aef168f0 2024-04-24T23:17:07.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15a97fa640 1 -- 172.21.15.112:0/2063831697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15bc019050 con 0x5587aef168f0 2024-04-24T23:17:07.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.174+0000 7f15a97fa640 1 -- 172.21.15.112:0/2063831697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00e620 con 0x5587aef168f0 2024-04-24T23:17:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 -- 172.21.15.112:0/2063831697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 msgr2=0x5587aef26ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26ac0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00fea0 tx=0x7f15bc005bd0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 -- 172.21.15.112:0/2063831697 shutdown_connections 2024-04-24T23:17:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef168f0 0x5587aef26ac0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef13130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 --2- 172.21.15.112:0/2063831697 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef10d90 0x5587aef12bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 -- 172.21.15.112:0/2063831697 >> 172.21.15.112:0/2063831697 conn(0x5587aeed8950 msgr2=0x5587aef1f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 -- 172.21.15.112:0/2063831697 shutdown_connections 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 -- 172.21.15.112:0/2063831697 wait complete. 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 -- start start 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.175+0000 7f15ccc33740 1 Processor -- start 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef23cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef24230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15ccc33740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef168f0 0x5587aef25240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15ccc33740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41a98f0 con 0x5587aef10d90 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15ccc33740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41a91f0 con 0x5587aef168f0 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15ccc33740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15c41a63d0 con 0x5587aef11780 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15ccc33740 10 client.0 _ll_register_callbacks cb 0x7f15bc00ff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef23cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef23cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48578/0 (socket says 172.21.15.112:48578) 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 -- 172.21.15.112:0/1839956578 learned_addr learned my addr 172.21.15.112:0/1839956578 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef24230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c3fff640 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef168f0 0x5587aef25240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 -- 172.21.15.112:0/1839956578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 msgr2=0x5587aef24230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef24230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 -- 172.21.15.112:0/1839956578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef168f0 msgr2=0x5587aef25240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef168f0 0x5587aef25240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 -- 172.21.15.112:0/1839956578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587aef25830 con 0x5587aef10d90 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15c37fe640 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef23cf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f15b80314d0 tx=0x7f15b8031bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800b435946ba519c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15a2ffd640 1 -- 172.21.15.112:0/1839956578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b8053070 con 0x5587aef10d90 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15a2ffd640 1 -- 172.21.15.112:0/1839956578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f15b804f3a0 con 0x5587aef10d90 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.176+0000 7f15ccc33740 1 -- 172.21.15.112:0/1839956578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15bc019050 con 0x5587aef10d90 2024-04-24T23:17:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15a2ffd640 1 -- 172.21.15.112:0/1839956578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f15b805f050 con 0x5587aef10d90 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15ccc33740 10 client.16266 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15ccc33740 1 -- 172.21.15.112:0/1839956578 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5587aef26be0 con 0x5587aef10d90 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15a2ffd640 1 -- 172.21.15.112:0/1839956578 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f15bc019050 con 0x5587aef10d90 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15a2ffd640 1 -- 172.21.15.112:0/1839956578 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7f15b8003c90 con 0x5587aef10d90 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15ccc33740 10 client.16266 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15ccc33740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15ccc33740 20 client.16266 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5587aef91950 0x5587aef93d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15ccc33740 4 client.16266 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15ccc33740 1 -- 172.21.15.112:0/1839956578 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587aee81800 con 0x5587aef91950 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5587aef91950 0x5587aef93d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15a2ffd640 10 client.16266 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.177+0000 7f15c2ffd640 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5587aef91950 0x5587aef93d40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4c1ffcac60784f server_cookie=fbf2d88c190652b0 in_seq=0 out_seq=0 2024-04-24T23:17:07.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.222+0000 7f15a2ffd640 1 -- 172.21.15.112:0/1839956578 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587aee81800 con 0x5587aef91950 2024-04-24T23:17:07.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15a2ffd640 10 client.16266 handle_command_reply: tid=0 2024-04-24T23:17:07.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 client.16266 shutdown 2024-04-24T23:17:07.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 -- 172.21.15.112:0/1839956578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 msgr2=0x5587aef23cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef23cf0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f15b80314d0 tx=0x7f15b8031bb0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 -- 172.21.15.112:0/1839956578 shutdown_connections 2024-04-24T23:17:07.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x5587aef91950 0x5587aef93d40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5587aef168f0 0x5587aef25240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15a2ffd640 0 client.16266 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587aef11780 0x5587aef24230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 --2- 172.21.15.112:0/1839956578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5587aef10d90 0x5587aef23cf0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 -- 172.21.15.112:0/1839956578 >> 172.21.15.112:0/1839956578 conn(0x5587aeed8950 msgr2=0x5587aef1f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 -- 172.21.15.112:0/1839956578 shutdown_connections 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 1 -- 172.21.15.112:0/1839956578 wait complete. 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.223+0000 7f15ccc33740 20 client.16266 trim_cache size 0 max 16384 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:07.226 INFO:teuthology.orchestra.run.smithi112.stdout: "return_code": 0, 2024-04-24T23:17:07.227 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_tag": "fa0c0b5b-d40c-4b66-bd90-abe8f0e4c76b", 2024-04-24T23:17:07.227 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": "asynchronous" 2024-04-24T23:17:07.227 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 -- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15ac0a2b90 msgr2=0x7f15ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15ac0a2b90 0x7f15ac0a5050 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009fd0 tx=0x7f15bc00fc00 comp rx=0 tx=0).stop 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 -- 172.21.15.112:0/3816457224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c4101870 msgr2=0x7f15c4197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c4101870 0x7f15c4197010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009ef0 tx=0x7f15b400ed10 comp rx=0 tx=0).stop 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 -- 172.21.15.112:0/3816457224 shutdown_connections 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f15ac0a2b90 0x7f15ac0a5050 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15c4102230 0x7f15c419e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15c4101870 0x7f15c4197010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 --2- 172.21.15.112:0/3816457224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15c4100680 0x7f15c4196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 -- 172.21.15.112:0/3816457224 >> 172.21.15.112:0/3816457224 conn(0x7f15c40fbbf0 msgr2=0x7f15c4104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 -- 172.21.15.112:0/3816457224 shutdown_connections 2024-04-24T23:17:07.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.224+0000 7f15c9e53640 1 -- 172.21.15.112:0/3816457224 wait complete. 2024-04-24T23:17:07.236 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:06 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:07.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:06 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:07.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:17:07.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:17:07.317 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:06 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 -- 172.21.15.112:0/99814541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400104b30 msgr2=0x7fd400106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 --2- 172.21.15.112:0/99814541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400104b30 0x7fd400106f20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009900 tx=0x7fd3f002a410 comp rx=0 tx=0).stop 2024-04-24T23:17:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 -- 172.21.15.112:0/99814541 shutdown_connections 2024-04-24T23:17:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 --2- 172.21.15.112:0/99814541 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd400107460 0x7fd4001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 --2- 172.21.15.112:0/99814541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400104b30 0x7fd400106f20 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 --2- 172.21.15.112:0/99814541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400102200 0x7fd4001045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 -- 172.21.15.112:0/99814541 >> 172.21.15.112:0/99814541 conn(0x7fd4000fbbf0 msgr2=0x7fd4000fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 -- 172.21.15.112:0/99814541 shutdown_connections 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.343+0000 7fd40859e640 1 -- 172.21.15.112:0/99814541 wait complete. 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40859e640 1 Processor -- start 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40859e640 1 -- start start 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40859e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400102200 0x7fd400101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40859e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400104b30 0x7fd4001001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40859e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd400107460 0x7fd4001006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40859e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010d530 con 0x7fd400102200 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40859e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010d3b0 con 0x7fd400107460 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40859e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd40010d6b0 con 0x7fd400104b30 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40633a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400102200 0x7fd400101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40633a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400102200 0x7fd400101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48604/0 (socket says 172.21.15.112:48604) 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd40633a640 1 -- 172.21.15.112:0/3460898513 learned_addr learned my addr 172.21.15.112:0/3460898513 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd405b39640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400104b30 0x7fd4001001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.344+0000 7fd406b3b640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd400107460 0x7fd4001006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd405b39640 1 -- 172.21.15.112:0/3460898513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd400107460 msgr2=0x7fd4001006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd405b39640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd400107460 0x7fd4001006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd405b39640 1 -- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400102200 msgr2=0x7fd400101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd405b39640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400102200 0x7fd400101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd405b39640 1 -- 172.21.15.112:0/3460898513 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd400100f80 con 0x7fd400104b30 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd40633a640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400102200 0x7fd400101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd405b39640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400104b30 0x7fd4001001b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009230 tx=0x7fd3f00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd3f77fe640 1 -- 172.21.15.112:0/3460898513 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f003f070 con 0x7fd400104b30 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd3f77fe640 1 -- 172.21.15.112:0/3460898513 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd3f003b460 con 0x7fd400104b30 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd40859e640 1 -- 172.21.15.112:0/3460898513 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4001a1cc0 con 0x7fd400104b30 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd3f77fe640 1 -- 172.21.15.112:0/3460898513 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0035070 con 0x7fd400104b30 2024-04-24T23:17:07.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.345+0000 7fd40859e640 1 -- 172.21.15.112:0/3460898513 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4001a2180 con 0x7fd400104b30 2024-04-24T23:17:07.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.347+0000 7fd3f77fe640 1 -- 172.21.15.112:0/3460898513 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd3f003b610 con 0x7fd400104b30 2024-04-24T23:17:07.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.348+0000 7fd3f77fe640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd3ec0a2c10 0x7fd3ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.348+0000 7fd40633a640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd3ec0a2c10 0x7fd3ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.348+0000 7fd3f77fe640 1 -- 172.21.15.112:0/3460898513 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd3f00ef9b0 con 0x7fd400104b30 2024-04-24T23:17:07.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.348+0000 7fd40633a640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd3ec0a2c10 0x7fd3ec0a50d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc011500 tx=0x7fd3fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40b386740 1 Processor -- start 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40b386740 1 -- start start 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40b386740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe38f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40b386740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38fc30 0x563fbe394730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40b386740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe394da0 0x563fbe397230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40b386740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563fbe2f36b0 con 0x563fbe38f240 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40b386740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563fbe29b880 con 0x563fbe38fc30 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40b386740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563fbe2ef670 con 0x563fbe394da0 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40633a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe38f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40633a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe38f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48612/0 (socket says 172.21.15.112:48612) 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd40633a640 1 -- 172.21.15.112:0/4194631114 learned_addr learned my addr 172.21.15.112:0/4194631114 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd405b39640 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38fc30 0x563fbe394730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.349+0000 7fd406b3b640 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe394da0 0x563fbe397230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd40633a640 1 -- 172.21.15.112:0/4194631114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe394da0 msgr2=0x563fbe397230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd40633a640 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe394da0 0x563fbe397230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd40633a640 1 -- 172.21.15.112:0/4194631114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38fc30 msgr2=0x563fbe394730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd40633a640 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38fc30 0x563fbe394730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd40633a640 1 -- 172.21.15.112:0/4194631114 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fbe0d17d0 con 0x563fbe38f240 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd406b3b640 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe394da0 0x563fbe397230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd405b39640 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38fc30 0x563fbe394730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd40633a640 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe38f660 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00f7c0 tx=0x7fd3fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f901375599c2916 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd3f4bfe640 1 -- 172.21.15.112:0/4194631114 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc02b070 con 0x563fbe38f240 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd3f4bfe640 1 -- 172.21.15.112:0/4194631114 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd3fc026420 con 0x563fbe38f240 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.350+0000 7fd3f4bfe640 1 -- 172.21.15.112:0/4194631114 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc01a5f0 con 0x563fbe38f240 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 -- 172.21.15.112:0/4194631114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 msgr2=0x563fbe38f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe38f660 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00f7c0 tx=0x7fd3fc014000 comp rx=0 tx=0).stop 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 -- 172.21.15.112:0/4194631114 shutdown_connections 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe394da0 0x563fbe397230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38fc30 0x563fbe394730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 --2- 172.21.15.112:0/4194631114 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe38f660 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 -- 172.21.15.112:0/4194631114 >> 172.21.15.112:0/4194631114 conn(0x563fbe38a8b0 msgr2=0x563fbe38acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 -- 172.21.15.112:0/4194631114 shutdown_connections 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 -- 172.21.15.112:0/4194631114 wait complete. 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 -- start start 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.351+0000 7fd40b386740 1 Processor -- start 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40b386740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38f240 0x563fbe399750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40b386740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 0x563fbe399c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40b386740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe394da0 0x563fbe3a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40b386740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563fbe397d50 con 0x563fbe394da0 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40b386740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x563fbe2aef50 con 0x563fbe38f240 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40b386740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563fbe397ed0 con 0x563fbe38fc30 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40b386740 10 client.0 _ll_register_callbacks cb 0x563fbe2ea000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:07.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd406b3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe394da0 0x563fbe3a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd406b3b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe394da0 0x563fbe3a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48618/0 (socket says 172.21.15.112:48618) 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40633a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38f240 0x563fbe399750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd406b3b640 1 -- 172.21.15.112:0/1521428273 learned_addr learned my addr 172.21.15.112:0/1521428273 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd405b39640 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 0x563fbe399c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40633a640 1 -- 172.21.15.112:0/1521428273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 msgr2=0x563fbe399c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40633a640 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 0x563fbe399c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40633a640 1 -- 172.21.15.112:0/1521428273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe394da0 msgr2=0x563fbe3a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40633a640 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe394da0 0x563fbe3a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40633a640 1 -- 172.21.15.112:0/1521428273 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fbe0073f0 con 0x563fbe38f240 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd406b3b640 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe394da0 0x563fbe3a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd40633a640 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38f240 0x563fbe399750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc011d90 tx=0x7fd3fc01ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4dc9aaeaaefcf3e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.352+0000 7fd3e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.150:3300/0 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/1521428273 <== mon.2 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc034060 con 0x563fbe38f240 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 1 -- 172.21.15.112:0/1521428273 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3fc026420 con 0x563fbe38f240 2024-04-24T23:17:07.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/1521428273 <== mon.2 v2:172.21.15.150:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd3fc005050 con 0x563fbe38f240 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/1521428273 <== mon.2 v2:172.21.15.150:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd3fc03b050 con 0x563fbe38f240 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 10 client.25664 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 1 -- 172.21.15.112:0/1521428273 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563fbdfc70f0 con 0x563fbe38f240 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/1521428273 <== mon.2 v2:172.21.15.150:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd3fc026420 con 0x563fbe38f240 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/1521428273 <== mon.2 v2:172.21.15.150:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7fd3fc004790 con 0x563fbe38f240 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 10 client.25664 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 10 client.25664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 20 client.25664 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563fbe3a50c0 0x563fbe3a7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 4 client.25664 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd40b386740 1 -- 172.21.15.112:0/1521428273 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fbe300bf0 con 0x563fbe3a50c0 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.353+0000 7fd405b39640 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563fbe3a50c0 0x563fbe3a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.354+0000 7fd405b39640 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563fbe3a50c0 0x563fbe3a7520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc52008ce60a250 server_cookie=8c4f375a183b697b in_seq=0 out_seq=0 2024-04-24T23:17:07.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.354+0000 7fd3e9ffb640 10 client.25664 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/1521428273 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fbe300bf0 con 0x563fbe3a50c0 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd3e9ffb640 10 client.25664 handle_command_reply: tid=0 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 client.25664 shutdown 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 -- 172.21.15.112:0/1521428273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38f240 msgr2=0x563fbe399750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38f240 0x563fbe399750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc011d90 tx=0x7fd3fc01ce60 comp rx=0 tx=0).stop 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 -- 172.21.15.112:0/1521428273 shutdown_connections 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563fbe3a50c0 0x563fbe3a7520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe394da0 0x563fbe3a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd3e9ffb640 0 client.25664 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 0x563fbe399c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 --2- 172.21.15.112:0/1521428273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe38f240 0x563fbe399750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 -- 172.21.15.112:0/1521428273 >> 172.21.15.112:0/1521428273 conn(0x563fbe38a8b0 msgr2=0x563fbe389af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 -- 172.21.15.112:0/1521428273 shutdown_connections 2024-04-24T23:17:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 1 -- 172.21.15.112:0/1521428273 wait complete. 2024-04-24T23:17:07.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.355+0000 7fd40b386740 20 client.25664 trim_cache size 0 max 16384 2024-04-24T23:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.374+0000 7fd40b386740 1 Processor -- start 2024-04-24T23:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.374+0000 7fd40b386740 1 -- start start 2024-04-24T23:17:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40b386740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe390490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40b386740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 0x563fbe3a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40b386740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe399a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40b386740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001ac8c0 con 0x563fbe38f240 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40b386740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001ae520 con 0x563fbe394da0 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40b386740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001acf90 con 0x563fbe38fc30 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe390490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe390490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48634/0 (socket says 172.21.15.112:48634) 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 -- 172.21.15.112:0/2373268864 learned_addr learned my addr 172.21.15.112:0/2373268864 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd406b3b640 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe399a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd405b39640 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 0x563fbe3a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 -- 172.21.15.112:0/2373268864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 msgr2=0x563fbe3a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 0x563fbe3a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 -- 172.21.15.112:0/2373268864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 msgr2=0x563fbe399a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe399a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 -- 172.21.15.112:0/2373268864 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fbe38b620 con 0x563fbe38f240 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd40633a640 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe390490 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc011e30 tx=0x7fd3fc00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35903052c4692b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd3ebfff640 1 -- 172.21.15.112:0/2373268864 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc034040 con 0x563fbe38f240 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd3ebfff640 1 -- 172.21.15.112:0/2373268864 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd3fc01c7e0 con 0x563fbe38f240 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.375+0000 7fd3ebfff640 1 -- 172.21.15.112:0/2373268864 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc02b070 con 0x563fbe38f240 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 -- 172.21.15.112:0/2373268864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 msgr2=0x563fbe390490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe390490 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc011e30 tx=0x7fd3fc00f710 comp rx=0 tx=0).stop 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 -- 172.21.15.112:0/2373268864 shutdown_connections 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe399a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38fc30 0x563fbe3a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 --2- 172.21.15.112:0/2373268864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38f240 0x563fbe390490 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 -- 172.21.15.112:0/2373268864 >> 172.21.15.112:0/2373268864 conn(0x563fbe38a8b0 msgr2=0x563fbe399360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 -- 172.21.15.112:0/2373268864 shutdown_connections 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 -- 172.21.15.112:0/2373268864 wait complete. 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.376+0000 7fd40b386740 1 -- start start 2024-04-24T23:17:07.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.377+0000 7fd40b386740 1 Processor -- start 2024-04-24T23:17:07.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd40b386740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38f240 0x563fbe39e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd40b386740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38fc30 0x563fbe39e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd40b386740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe39f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd40b386740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001add90 con 0x563fbe38fc30 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd40b386740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001ad690 con 0x563fbe394da0 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd40b386740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001aa870 con 0x563fbe38f240 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd40b386740 10 client.0 _ll_register_callbacks cb 0x7fd3fc029070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd406b3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe39f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd405b39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38fc30 0x563fbe39e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd406b3b640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe39f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.112:50448/0 (socket says 172.21.15.112:50448) 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd406b3b640 1 -- 172.21.15.112:0/2800491905 learned_addr learned my addr 172.21.15.112:0/2800491905 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd405b39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38fc30 0x563fbe39e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48636/0 (socket says 172.21.15.112:48636) 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.378+0000 7fd40633a640 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38f240 0x563fbe39e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd405b39640 1 -- 172.21.15.112:0/2800491905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38f240 msgr2=0x563fbe39e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd405b39640 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38f240 0x563fbe39e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd405b39640 1 -- 172.21.15.112:0/2800491905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 msgr2=0x563fbe39f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd405b39640 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe39f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd405b39640 1 -- 172.21.15.112:0/2800491905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fbe39fb40 con 0x563fbe38fc30 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd40633a640 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38f240 0x563fbe39e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd405b39640 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38fc30 0x563fbe39e540 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd3f002a930 tx=0x7fd3f002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb61ae140654515c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd3e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/2800491905 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0048040 con 0x563fbe38fc30 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/2800491905 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fd3f003ba10 con 0x563fbe38fc30 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd40b386740 1 -- 172.21.15.112:0/2800491905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3fc01c7e0 con 0x563fbe38fc30 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/2800491905 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd3f00bc130 con 0x563fbe38fc30 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd40b386740 10 client.16284 fetch_fsmap learned FSMap version 35 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd40b386740 1 -- 172.21.15.112:0/2800491905 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563fbe3a0f20 con 0x563fbe38fc30 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/2800491905 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fd3fc01c7e0 con 0x563fbe38fc30 2024-04-24T23:17:07.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.379+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/2800491905 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 35) ==== 8394+0+0 (secure 0 0 0) 0x7fd3f00c4600 con 0x563fbe38fc30 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd40b386740 10 client.16284 fetch_fsmap finished waiting for FSMap version 35 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd40b386740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd40b386740 20 client.16284 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd40b386740 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563fbe40bde0 0x563fbe40e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd40b386740 4 client.16284 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd40b386740 1 -- 172.21.15.112:0/2800491905 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fbe2faf40 con 0x563fbe40bde0 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd40633a640 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563fbe40bde0 0x563fbe40e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd3e9ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:07.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.380+0000 7fd40633a640 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563fbe40bde0 0x563fbe40e200 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3667eb3ee84da3 server_cookie=5a7408dd86154be8 in_seq=0 out_seq=0 2024-04-24T23:17:07.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd3e9ffb640 1 -- 172.21.15.112:0/2800491905 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x563fbe2faf40 con 0x563fbe40bde0 2024-04-24T23:17:07.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd3e9ffb640 10 client.16284 handle_command_reply: tid=0 2024-04-24T23:17:07.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 client.16284 shutdown 2024-04-24T23:17:07.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 -- 172.21.15.112:0/2800491905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38fc30 msgr2=0x563fbe39e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38fc30 0x563fbe39e540 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd3f002a930 tx=0x7fd3f002a410 comp rx=0 tx=0).stop 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 -- 172.21.15.112:0/2800491905 shutdown_connections 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x563fbe40bde0 0x563fbe40e200 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x563fbe394da0 0x563fbe39f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563fbe38fc30 0x563fbe39e540 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd3e9ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 --2- 172.21.15.112:0/2800491905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fbe38f240 0x563fbe39e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 -- 172.21.15.112:0/2800491905 >> 172.21.15.112:0/2800491905 conn(0x563fbe38a8b0 msgr2=0x563fbe3992d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 -- 172.21.15.112:0/2800491905 shutdown_connections 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 1 -- 172.21.15.112:0/2800491905 wait complete. 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.383+0000 7fd40b386740 20 client.16284 trim_cache size 0 max 16384 2024-04-24T23:17:07.388 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "scrub active (181 inodes in the stack)", 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": { 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "fa0c0b5b-d40c-4b66-bd90-abe8f0e4c76b": { 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "path": "/", 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tag": "fa0c0b5b-d40c-4b66-bd90-abe8f0e4c76b", 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: "options": "recursive,force" 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:07.389 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:07.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 -- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd3ec0a2c10 msgr2=0x7fd3ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd3ec0a2c10 0x7fd3ec0a50d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc011500 tx=0x7fd3fc005f90 comp rx=0 tx=0).stop 2024-04-24T23:17:07.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 -- 172.21.15.112:0/3460898513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400104b30 msgr2=0x7fd4001001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:07.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400104b30 0x7fd4001001b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009230 tx=0x7fd3f00042f0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 -- 172.21.15.112:0/3460898513 shutdown_connections 2024-04-24T23:17:07.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fd3ec0a2c10 0x7fd3ec0a50d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd400107460 0x7fd4001006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd400104b30 0x7fd4001001b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 --2- 172.21.15.112:0/3460898513 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd400102200 0x7fd400101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:07.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 -- 172.21.15.112:0/3460898513 >> 172.21.15.112:0/3460898513 conn(0x7fd4000fbbf0 msgr2=0x7fd400102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:07.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 -- 172.21.15.112:0/3460898513 shutdown_connections 2024-04-24T23:17:07.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:07.384+0000 7fd40859e640 1 -- 172.21.15.112:0/3460898513 wait complete. 2024-04-24T23:17:07.396 INFO:tasks.cephfs.filesystem:scrub status for tag:fa0c0b5b-d40c-4b66-bd90-abe8f0e4c76b - {'path': '/', 'tag': 'fa0c0b5b-d40c-4b66-bd90-abe8f0e4c76b', 'options': 'recursive,force'} 2024-04-24T23:17:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:07 smithi187 ceph-mon[33292]: pgmap v434: 129 pgs: 129 active+clean; 183 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-04-24T23:17:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:07 smithi187 ceph-mon[33292]: scrub queued for path: / 2024-04-24T23:17:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:07 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:07 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:07 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:17:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:07 smithi112 ceph-mon[29186]: pgmap v434: 129 pgs: 129 active+clean; 183 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-04-24T23:17:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:07 smithi112 ceph-mon[29186]: scrub queued for path: / 2024-04-24T23:17:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:07 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:07 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:08.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:07 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:17:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:07 smithi150 ceph-mon[32247]: pgmap v434: 129 pgs: 129 active+clean; 183 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-04-24T23:17:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:07 smithi150 ceph-mon[32247]: scrub queued for path: / 2024-04-24T23:17:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:07 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:07 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:08.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:07 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:17:08.576 INFO:tasks.workunit.client.0.smithi112.stderr:+ sync 2024-04-24T23:17:09.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:08 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:09.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:08 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:09.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:08 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:09.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:08 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:09.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:08 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:09.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:08 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:09.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:08 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:09.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:08 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:09.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:08 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:09.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:08 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:09.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:08 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:09.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:08 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:09.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:08 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:09.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:08 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:09.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:08 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:09.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:08 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:09.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:08 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:09.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:08 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.219 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:10.219 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.219 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:10.219 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:10.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:10.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: active paths [/] 2024-04-24T23:17:10.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: pgmap v435: 129 pgs: 129 active+clean; 189 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-04-24T23:17:10.220 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:09 smithi150 ceph-mon[32247]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:10.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: active paths [/] 2024-04-24T23:17:10.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: pgmap v435: 129 pgs: 129 active+clean; 189 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-04-24T23:17:10.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:09 smithi187 ceph-mon[33292]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:10.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: active paths [/] 2024-04-24T23:17:10.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:10.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: pgmap v435: 129 pgs: 129 active+clean; 189 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 310 op/s 2024-04-24T23:17:10.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:09 smithi112 ceph-mon[29186]: scrub summary: idle+waiting paths [/] 2024-04-24T23:17:11.262 INFO:tasks.workunit.client.0.smithi112.stderr:+ tar c linux-4.0.5 2024-04-24T23:17:11.996 INFO:tasks.workunit.client.0.smithi112.stdout:Ok 2024-04-24T23:17:11.996 INFO:tasks.workunit.client.0.smithi112.stderr:+ echo Ok 2024-04-24T23:17:11.997 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T23:17:11.997 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T23:17:12.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:11 smithi187 ceph-mon[33292]: pgmap v436: 129 pgs: 129 active+clean; 207 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 472 op/s 2024-04-24T23:17:12.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:11 smithi112 ceph-mon[29186]: pgmap v436: 129 pgs: 129 active+clean; 207 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 472 op/s 2024-04-24T23:17:12.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:11 smithi150 ceph-mon[32247]: pgmap v436: 129 pgs: 129 active+clean; 207 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 472 op/s 2024-04-24T23:17:14.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:13 smithi187 ceph-mon[33292]: pgmap v437: 129 pgs: 129 active+clean; 210 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-04-24T23:17:14.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:13 smithi112 ceph-mon[29186]: pgmap v437: 129 pgs: 129 active+clean; 210 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-04-24T23:17:14.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:13 smithi150 ceph-mon[32247]: pgmap v437: 129 pgs: 129 active+clean; 210 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.1 MiB/s wr, 379 op/s 2024-04-24T23:17:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:15 smithi187 ceph-mon[33292]: pgmap v438: 129 pgs: 129 active+clean; 197 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 392 op/s 2024-04-24T23:17:16.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:15 smithi112 ceph-mon[29186]: pgmap v438: 129 pgs: 129 active+clean; 197 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 392 op/s 2024-04-24T23:17:16.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:15 smithi150 ceph-mon[32247]: pgmap v438: 129 pgs: 129 active+clean; 197 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 392 op/s 2024-04-24T23:17:17.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:16 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:17:17.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:16 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:17:17.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:16 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:17:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:16 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:17:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:16 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:17:17.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:16 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:17:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:16 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:17:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:16 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:17:17.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:16 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:17:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:17 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:17:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:17 smithi187 ceph-mon[33292]: scrub summary: idle 2024-04-24T23:17:18.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:17 smithi187 ceph-mon[33292]: pgmap v439: 129 pgs: 129 active+clean; 131 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 432 op/s 2024-04-24T23:17:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:17 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:17:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:17 smithi112 ceph-mon[29186]: scrub summary: idle 2024-04-24T23:17:18.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:17 smithi112 ceph-mon[29186]: pgmap v439: 129 pgs: 129 active+clean; 131 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 432 op/s 2024-04-24T23:17:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:17 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:17:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:17 smithi150 ceph-mon[32247]: scrub summary: idle 2024-04-24T23:17:18.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:17 smithi150 ceph-mon[32247]: pgmap v439: 129 pgs: 129 active+clean; 131 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 432 op/s 2024-04-24T23:17:20.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:19 smithi187 ceph-mon[33292]: pgmap v440: 129 pgs: 129 active+clean; 116 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-04-24T23:17:20.278 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:19 smithi112 ceph-mon[29186]: pgmap v440: 129 pgs: 129 active+clean; 116 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-04-24T23:17:20.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:19 smithi150 ceph-mon[32247]: pgmap v440: 129 pgs: 129 active+clean; 116 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-04-24T23:17:22.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:21 smithi187 ceph-mon[33292]: pgmap v441: 129 pgs: 129 active+clean; 118 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 448 op/s 2024-04-24T23:17:22.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:21 smithi112 ceph-mon[29186]: pgmap v441: 129 pgs: 129 active+clean; 118 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 448 op/s 2024-04-24T23:17:22.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:21 smithi150 ceph-mon[32247]: pgmap v441: 129 pgs: 129 active+clean; 118 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 448 op/s 2024-04-24T23:17:24.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:23 smithi187 ceph-mon[33292]: pgmap v442: 129 pgs: 129 active+clean; 117 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-24T23:17:24.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:23 smithi112 ceph-mon[29186]: pgmap v442: 129 pgs: 129 active+clean; 117 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-24T23:17:24.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:23 smithi150 ceph-mon[32247]: pgmap v442: 129 pgs: 129 active+clean; 117 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-24T23:17:24.446 INFO:tasks.workunit:Stopping ['fs/norstats'] on client.0... 2024-04-24T23:17:24.447 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T23:17:24.732 DEBUG:teuthology.parallel:result is None 2024-04-24T23:17:24.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:17:24.768 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:17:24.768 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-24T23:17:24.778 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-24T23:17:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:25 smithi187 ceph-mon[33292]: pgmap v443: 129 pgs: 129 active+clean; 114 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-04-24T23:17:26.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:25 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:25 smithi112 ceph-mon[29186]: pgmap v443: 129 pgs: 129 active+clean; 114 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-04-24T23:17:26.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:25 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:25 smithi150 ceph-mon[32247]: pgmap v443: 129 pgs: 129 active+clean; 114 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 296 op/s 2024-04-24T23:17:26.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:25 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:28.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:27 smithi112 ceph-mon[29186]: pgmap v444: 129 pgs: 129 active+clean; 111 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 379 op/s 2024-04-24T23:17:28.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:27 smithi150 ceph-mon[32247]: pgmap v444: 129 pgs: 129 active+clean; 111 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 379 op/s 2024-04-24T23:17:28.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:27 smithi187 ceph-mon[33292]: pgmap v444: 129 pgs: 129 active+clean; 111 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 379 op/s 2024-04-24T23:17:29.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:29 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:29.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:29 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:29.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:29 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:30.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:30 smithi112 ceph-mon[29186]: pgmap v445: 129 pgs: 129 active+clean; 109 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-24T23:17:30.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:30 smithi150 ceph-mon[32247]: pgmap v445: 129 pgs: 129 active+clean; 109 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-24T23:17:30.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:30 smithi187 ceph-mon[33292]: pgmap v445: 129 pgs: 129 active+clean; 109 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 328 op/s 2024-04-24T23:17:31.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:31 smithi112 ceph-mon[29186]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:31.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:31 smithi150 ceph-mon[32247]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:31.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:31 smithi187 ceph-mon[33292]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-24T23:17:32.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:32 smithi112 ceph-mon[29186]: pgmap v446: 129 pgs: 129 active+clean; 109 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 977 KiB/s wr, 295 op/s 2024-04-24T23:17:32.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:32 smithi150 ceph-mon[32247]: pgmap v446: 129 pgs: 129 active+clean; 109 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 977 KiB/s wr, 295 op/s 2024-04-24T23:17:32.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:32 smithi187 ceph-mon[33292]: pgmap v446: 129 pgs: 129 active+clean; 109 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 977 KiB/s wr, 295 op/s 2024-04-24T23:17:34.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:34 smithi112 ceph-mon[29186]: pgmap v447: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 530 KiB/s wr, 197 op/s 2024-04-24T23:17:34.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:34 smithi150 ceph-mon[32247]: pgmap v447: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 530 KiB/s wr, 197 op/s 2024-04-24T23:17:34.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:34 smithi187 ceph-mon[33292]: pgmap v447: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 530 KiB/s wr, 197 op/s 2024-04-24T23:17:36.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:36 smithi112 ceph-mon[29186]: pgmap v448: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 528 KiB/s wr, 187 op/s 2024-04-24T23:17:36.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:36 smithi150 ceph-mon[32247]: pgmap v448: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 528 KiB/s wr, 187 op/s 2024-04-24T23:17:36.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:36 smithi187 ceph-mon[33292]: pgmap v448: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 528 KiB/s wr, 187 op/s 2024-04-24T23:17:37.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-24T23:17:37.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-24T23:17:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.508+0000 7fe70d9fb640 1 -- 172.21.15.112:0/3190279571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708101790 msgr2=0x7fe708101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.508+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/3190279571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708101790 0x7fe708101bb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009ca0 tx=0x7fe6f802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 -- 172.21.15.112:0/3190279571 shutdown_connections 2024-04-24T23:17:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/3190279571 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe708103340 0x7fe708109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/3190279571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe708102980 0x7fe708102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/3190279571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708101790 0x7fe708101bb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 -- 172.21.15.112:0/3190279571 >> 172.21.15.112:0/3190279571 conn(0x7fe7080fcd20 msgr2=0x7fe7080ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 -- 172.21.15.112:0/3190279571 shutdown_connections 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 -- 172.21.15.112:0/3190279571 wait complete. 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 Processor -- start 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.509+0000 7fe70d9fb640 1 -- start start 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70d9fb640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe708101790 0x7fe708197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70d9fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe708102980 0x7fe708198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70d9fb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708103340 0x7fe70819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70d9fb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe70810d5a0 con 0x7fe708103340 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70d9fb640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe70810d420 con 0x7fe708101790 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70d9fb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe70810d720 con 0x7fe708102980 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708103340 0x7fe70819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708103340 0x7fe70819f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54546/0 (socket says 172.21.15.112:54546) 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 -- 172.21.15.112:0/1822032783 learned_addr learned my addr 172.21.15.112:0/1822032783 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe706ffd640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe708101790 0x7fe708197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe708102980 0x7fe708198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 -- 172.21.15.112:0/1822032783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe708102980 msgr2=0x7fe708198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe708102980 0x7fe708198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 -- 172.21.15.112:0/1822032783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe708101790 msgr2=0x7fe708197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe708101790 0x7fe708197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 -- 172.21.15.112:0/1822032783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70819f850 con 0x7fe708103340 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe708102980 0x7fe708198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe7077fe640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708103340 0x7fe70819f1b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe70000d750 tx=0x7fe70000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70c9f9640 1 -- 172.21.15.112:0/1822032783 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700017070 con 0x7fe708103340 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70d9fb640 1 -- 172.21.15.112:0/1822032783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70819fb40 con 0x7fe708103340 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70c9f9640 1 -- 172.21.15.112:0/1822032783 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe700005520 con 0x7fe708103340 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70c9f9640 1 -- 172.21.15.112:0/1822032783 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700004e00 con 0x7fe708103340 2024-04-24T23:17:37.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.510+0000 7fe70d9fb640 1 -- 172.21.15.112:0/1822032783 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7081a0090 con 0x7fe708103340 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.512+0000 7fe70c9f9640 1 -- 172.21.15.112:0/1822032783 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe700004fa0 con 0x7fe708103340 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7107df740 1 Processor -- start 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7107df740 1 -- start start 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7107df740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7107df740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7c30 0x55c5322ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7107df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322acda0 0x55c5322af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7107df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c53220b6b0 con 0x55c5322acda0 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7107df740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c5321b3880 con 0x55c5322a7c30 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7107df740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c532207670 con 0x55c5322a7240 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe70c9f9640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6f00a2b90 0x7fe6f00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7077fe640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6f00a2b90 0x7fe6f00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322acda0 0x55c5322af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322acda0 0x55c5322af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54562/0 (socket says 172.21.15.112:54562) 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7067fc640 1 -- 172.21.15.112:0/2373702050 learned_addr learned my addr 172.21.15.112:0/2373702050 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe70c9f9640 1 -- 172.21.15.112:0/1822032783 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fe7000ceed0 con 0x7fe708103340 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7067fc640 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe706ffd640 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7c30 0x55c5322ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7067fc640 1 -- 172.21.15.112:0/2373702050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 msgr2=0x55c5322a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7067fc640 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7067fc640 1 -- 172.21.15.112:0/2373702050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7c30 msgr2=0x55c5322ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.513+0000 7fe7067fc640 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7c30 0x55c5322ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.514+0000 7fe7067fc640 1 -- 172.21.15.112:0/2373702050 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c531fe97d0 con 0x55c5322acda0 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.514+0000 7fe706ffd640 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7c30 0x55c5322ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.514+0000 7fe7067fc640 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322acda0 0x55c5322af230 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00adf0 tx=0x7fe6fc00a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f29081fa1052b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.514+0000 7fe7077fe640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6f00a2b90 0x7fe6f00a5050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe70009b2d0 tx=0x7fe700085e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.514+0000 7fe6edffb640 1 -- 172.21.15.112:0/2373702050 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc01c070 con 0x55c5322acda0 2024-04-24T23:17:37.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.514+0000 7fe6edffb640 1 -- 172.21.15.112:0/2373702050 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6fc00fdf0 con 0x55c5322acda0 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.514+0000 7fe6edffb640 1 -- 172.21.15.112:0/2373702050 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc014850 con 0x55c5322acda0 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 -- 172.21.15.112:0/2373702050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322acda0 msgr2=0x55c5322af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322acda0 0x55c5322af230 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00adf0 tx=0x7fe6fc00a550 comp rx=0 tx=0).stop 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 -- 172.21.15.112:0/2373702050 shutdown_connections 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322acda0 0x55c5322af230 secure :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00adf0 tx=0x7fe6fc00a550 comp rx=0 tx=0).stop 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7c30 0x55c5322ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 --2- 172.21.15.112:0/2373702050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 -- 172.21.15.112:0/2373702050 >> 172.21.15.112:0/2373702050 conn(0x55c5322a28b0 msgr2=0x55c5322a2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 -- 172.21.15.112:0/2373702050 shutdown_connections 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 -- 172.21.15.112:0/2373702050 wait complete. 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.515+0000 7fe7107df740 1 -- start start 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7107df740 1 Processor -- start 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7107df740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7107df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7107df740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 0x55c5322b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7107df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c5322afd50 con 0x55c5322a7c30 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7107df740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55c5321c6f50 con 0x55c5322b6df0 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7107df740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c5322afed0 con 0x55c5322a7240 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7107df740 10 client.0 _ll_register_callbacks cb 0x55c532202000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54568/0 (socket says 172.21.15.112:54568) 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 -- 172.21.15.112:0/1517812154 learned_addr learned my addr 172.21.15.112:0/1517812154 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7067fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 0x55c5322b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 -- 172.21.15.112:0/1517812154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 msgr2=0x55c5322b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 -- 172.21.15.112:0/1517812154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 msgr2=0x55c5322b91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 0x55c5322b91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 -- 172.21.15.112:0/1517812154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c531f1f3f0 con 0x55c5322a7c30 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe706ffd640 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322b1dc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8031ae0 tx=0x7fe6f8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb47de2bc97a3e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.516+0000 7fe6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1517812154 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f802f810 con 0x55c5322a7c30 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 1 -- 172.21.15.112:0/1517812154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6fc00fdf0 con 0x55c5322a7c30 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1517812154 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fe6f802f9b0 con 0x55c5322a7c30 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1517812154 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe6f8053050 con 0x55c5322a7c30 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 10 client.16296 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 1 -- 172.21.15.112:0/1517812154 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c531edf0f0 con 0x55c5322a7c30 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1517812154 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6fc00fdf0 con 0x55c5322a7c30 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1517812154 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7fe6f804e030 con 0x55c5322a7c30 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 10 client.16296 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 20 client.16296 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:37.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c5322bdc00 0x55c5322c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 4 client.16296 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7107df740 1 -- 172.21.15.112:0/1517812154 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c532218bf0 con 0x55c5322bdc00 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.517+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c5322bdc00 0x55c5322c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.518+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c5322bdc00 0x55c5322c0040 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=521086b0adb49b4b server_cookie=8e6f0220fa2fde2f in_seq=0 out_seq=0 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.518+0000 7fe6e6ffd640 10 client.16296 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.518+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1517812154 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c532218bf0 con 0x55c5322bdc00 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.518+0000 7fe6e6ffd640 10 client.16296 handle_command_reply: tid=0 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.518+0000 7fe7107df740 1 client.16296 shutdown 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 -- 172.21.15.112:0/1517812154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 msgr2=0x55c5322b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322b1dc0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8031ae0 tx=0x7fe6f8005fa0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 -- 172.21.15.112:0/1517812154 shutdown_connections 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c5322bdc00 0x55c5322c0040 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 0x55c5322b91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322b1dc0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe6e6ffd640 0 client.16296 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:37.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 --2- 172.21.15.112:0/1517812154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 -- 172.21.15.112:0/1517812154 >> 172.21.15.112:0/1517812154 conn(0x55c5322a28b0 msgr2=0x55c5322a1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 -- 172.21.15.112:0/1517812154 shutdown_connections 2024-04-24T23:17:37.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 1 -- 172.21.15.112:0/1517812154 wait complete. 2024-04-24T23:17:37.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.519+0000 7fe7107df740 20 client.16296 trim_cache size 0 max 16384 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7107df740 1 Processor -- start 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7107df740 1 -- start start 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7107df740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322a8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7107df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7107df740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 0x55c5322a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7107df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7081aa6b0 con 0x55c5322a7c30 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7107df740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe7081ac3d0 con 0x55c5322b6df0 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7107df740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7081aad80 con 0x55c5322a7240 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe706ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe706ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54572/0 (socket says 172.21.15.112:54572) 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe706ffd640 1 -- 172.21.15.112:0/2980474139 learned_addr learned my addr 172.21.15.112:0/2980474139 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:37.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.536+0000 7fe7067fc640 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe7067fc640 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 0x55c5322a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe706ffd640 1 -- 172.21.15.112:0/2980474139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 msgr2=0x55c5322a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe706ffd640 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322a8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe706ffd640 1 -- 172.21.15.112:0/2980474139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 msgr2=0x55c5322a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe706ffd640 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 0x55c5322a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe706ffd640 1 -- 172.21.15.112:0/2980474139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5322a8010 con 0x55c5322a7c30 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe706ffd640 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322a92b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe6f803b450 tx=0x7fe6f802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8805a26333bbf0af server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe6ed7fa640 1 -- 172.21.15.112:0/2980474139 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f803bb70 con 0x55c5322a7c30 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe6ed7fa640 1 -- 172.21.15.112:0/2980474139 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6f8036040 con 0x55c5322a7c30 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe6ed7fa640 1 -- 172.21.15.112:0/2980474139 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80043c0 con 0x55c5322a7c30 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe7107df740 1 -- 172.21.15.112:0/2980474139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 msgr2=0x55c5322a92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.537+0000 7fe7107df740 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322a92b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe6f803b450 tx=0x7fe6f802bfd0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 -- 172.21.15.112:0/2980474139 shutdown_connections 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322b6df0 0x55c5322a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322a7c30 0x55c5322a92b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 --2- 172.21.15.112:0/2980474139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7240 0x55c5322a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 -- 172.21.15.112:0/2980474139 >> 172.21.15.112:0/2980474139 conn(0x55c5322a28b0 msgr2=0x55c5322b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 -- 172.21.15.112:0/2980474139 shutdown_connections 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 -- 172.21.15.112:0/2980474139 wait complete. 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 -- start start 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 Processor -- start 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7240 0x55c5322cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7c30 0x55c5322cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322b6df0 0x55c5322d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7081abc40 con 0x55c5322b6df0 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe7081ab4e0 con 0x55c5322a7240 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7081a8660 con 0x55c5322a7c30 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7107df740 10 client.0 _ll_register_callbacks cb 0x7fe6f8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:37.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322b6df0 0x55c5322d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7067fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322b6df0 0x55c5322d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54574/0 (socket says 172.21.15.112:54574) 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7067fc640 1 -- 172.21.15.112:0/1249510363 learned_addr learned my addr 172.21.15.112:0/1249510363 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.538+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7240 0x55c5322cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe706ffd640 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7c30 0x55c5322cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7067fc640 1 -- 172.21.15.112:0/1249510363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7c30 msgr2=0x55c5322cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7c30 0x55c5322cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7067fc640 1 -- 172.21.15.112:0/1249510363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7240 msgr2=0x55c5322cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7240 0x55c5322cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7067fc640 1 -- 172.21.15.112:0/1249510363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5322d0bb0 con 0x55c5322b6df0 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe706ffd640 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7c30 0x55c5322cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7067fc640 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322b6df0 0x55c5322d05c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc018fd0 tx=0x7fe6fc02b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3c8a5dce517c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe6e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1249510363 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc01c040 con 0x55c5322b6df0 2024-04-24T23:17:37.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1249510363 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fe6fc032040 con 0x55c5322b6df0 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 1 -- 172.21.15.112:0/1249510363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f8036040 con 0x55c5322b6df0 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1249510363 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe6fc03e050 con 0x55c5322b6df0 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 10 client.16302 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 1 -- 172.21.15.112:0/1249510363 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c532324150 con 0x55c5322b6df0 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1249510363 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe6f8036040 con 0x55c5322b6df0 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1249510363 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7fe6fc022040 con 0x55c5322b6df0 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 10 client.16302 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 20 client.16302 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c532324940 0x55c532326d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 4 client.16302 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.539+0000 7fe7107df740 1 -- 172.21.15.112:0/1249510363 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c532212f40 con 0x55c532324940 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe706ffd640 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c532324940 0x55c532326d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe706ffd640 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c532324940 0x55c532326d60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43f0706a2d0808ed server_cookie=b5407b92e3e2b89a in_seq=0 out_seq=0 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe6e6ffd640 10 client.16302 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe6e6ffd640 1 -- 172.21.15.112:0/1249510363 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c532212f40 con 0x55c532324940 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe6e6ffd640 10 client.16302 handle_command_reply: tid=0 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 client.16302 shutdown 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 -- 172.21.15.112:0/1249510363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322b6df0 msgr2=0x55c5322d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322b6df0 0x55c5322d05c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc018fd0 tx=0x7fe6fc02b040 comp rx=0 tx=0).stop 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 -- 172.21.15.112:0/1249510363 shutdown_connections 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55c532324940 0x55c532326d60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c5322b6df0 0x55c5322d05c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5322a7c30 0x55c5322cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe6e6ffd640 0 client.16302 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 --2- 172.21.15.112:0/1249510363 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55c5322a7240 0x55c5322cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.540+0000 7fe7107df740 1 -- 172.21.15.112:0/1249510363 >> 172.21.15.112:0/1249510363 conn(0x55c5322a28b0 msgr2=0x55c5322a92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe7107df740 1 -- 172.21.15.112:0/1249510363 shutdown_connections 2024-04-24T23:17:37.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe7107df740 1 -- 172.21.15.112:0/1249510363 wait complete. 2024-04-24T23:17:37.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe7107df740 20 client.16302 trim_cache size 0 max 16384 2024-04-24T23:17:37.550 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:37.550 INFO:teuthology.orchestra.run.smithi112.stdout: "status": "no active scrubs running", 2024-04-24T23:17:37.550 INFO:teuthology.orchestra.run.smithi112.stdout: "scrubs": {} 2024-04-24T23:17:37.550 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:37.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 -- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6f00a2b90 msgr2=0x7fe6f00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6f00a2b90 0x7fe6f00a5050 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe70009b2d0 tx=0x7fe700085e30 comp rx=0 tx=0).stop 2024-04-24T23:17:37.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 -- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708103340 msgr2=0x7fe70819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708103340 0x7fe70819f1b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe70000d750 tx=0x7fe70000dc10 comp rx=0 tx=0).stop 2024-04-24T23:17:37.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 -- 172.21.15.112:0/1822032783 shutdown_connections 2024-04-24T23:17:37.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6f00a2b90 0x7fe6f00a5050 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe708103340 0x7fe70819f1b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe708102980 0x7fe708198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 --2- 172.21.15.112:0/1822032783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe708101790 0x7fe708197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 -- 172.21.15.112:0/1822032783 >> 172.21.15.112:0/1822032783 conn(0x7fe7080fcd20 msgr2=0x7fe708105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 -- 172.21.15.112:0/1822032783 shutdown_connections 2024-04-24T23:17:37.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.541+0000 7fe70d9fb640 1 -- 172.21.15.112:0/1822032783 wait complete. 2024-04-24T23:17:37.554 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-24T23:17:37.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:17:37.655 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 -- 172.21.15.112:0/2738531871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee34102250 msgr2=0x7fee34108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 --2- 172.21.15.112:0/2738531871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee34102250 0x7fee34108ae0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee30009ca0 tx=0x7fee3002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 -- 172.21.15.112:0/2738531871 shutdown_connections 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 --2- 172.21.15.112:0/2738531871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee34102250 0x7fee34108ae0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 --2- 172.21.15.112:0/2738531871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee34101890 0x7fee34101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 --2- 172.21.15.112:0/2738531871 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee341006a0 0x7fee34100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 -- 172.21.15.112:0/2738531871 >> 172.21.15.112:0/2738531871 conn(0x7fee340fbc30 msgr2=0x7fee340fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 -- 172.21.15.112:0/2738531871 shutdown_connections 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.651+0000 7fee3bda5640 1 -- 172.21.15.112:0/2738531871 wait complete. 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee3bda5640 1 Processor -- start 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee3bda5640 1 -- start start 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee3bda5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee341006a0 0x7fee34197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.656 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee3bda5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee34101890 0x7fee34198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee3bda5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee34102250 0x7fee3419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee3bda5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee3410c450 con 0x7fee341006a0 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee3bda5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fee3410c2d0 con 0x7fee34101890 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee3bda5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fee3410c5d0 con 0x7fee34102250 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39b41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee341006a0 0x7fee34197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39b41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee341006a0 0x7fee34197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54592/0 (socket says 172.21.15.112:54592) 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39b41640 1 -- 172.21.15.112:0/3138560684 learned_addr learned my addr 172.21.15.112:0/3138560684 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39340640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee34101890 0x7fee34198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39b41640 1 -- 172.21.15.112:0/3138560684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee34102250 msgr2=0x7fee3419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39b41640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee34102250 0x7fee3419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39b41640 1 -- 172.21.15.112:0/3138560684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee34101890 msgr2=0x7fee34198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39b41640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee34101890 0x7fee34198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39b41640 1 -- 172.21.15.112:0/3138560684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3419f770 con 0x7fee341006a0 2024-04-24T23:17:37.657 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.652+0000 7fee39340640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee34101890 0x7fee34198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.653+0000 7fee39b41640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee341006a0 0x7fee34197b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00e9c0 tx=0x7fee2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.653+0000 7fee2affd640 1 -- 172.21.15.112:0/3138560684 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00cd80 con 0x7fee341006a0 2024-04-24T23:17:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.653+0000 7fee2affd640 1 -- 172.21.15.112:0/3138560684 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fee2c0035f0 con 0x7fee341006a0 2024-04-24T23:17:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.653+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3419fa60 con 0x7fee341006a0 2024-04-24T23:17:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.653+0000 7fee2affd640 1 -- 172.21.15.112:0/3138560684 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c010690 con 0x7fee341006a0 2024-04-24T23:17:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.653+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3419ffb0 con 0x7fee341006a0 2024-04-24T23:17:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.654+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee341a0240 con 0x7fee341006a0 2024-04-24T23:17:37.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.655+0000 7fee2affd640 1 -- 172.21.15.112:0/3138560684 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fee2c0037a0 con 0x7fee341006a0 2024-04-24T23:17:37.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.655+0000 7fee2affd640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fee200a2b90 0x7fee200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.655+0000 7fee39340640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fee200a2b90 0x7fee200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.655+0000 7fee2affd640 1 -- 172.21.15.112:0/3138560684 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fee2c0cf9d0 con 0x7fee341006a0 2024-04-24T23:17:37.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.656+0000 7fee39340640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fee200a2b90 0x7fee200a5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee24005fd0 tx=0x7fee24005ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.660+0000 7fee2affd640 1 -- 172.21.15.112:0/3138560684 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fee2c09c540 con 0x7fee341006a0 2024-04-24T23:17:37.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.872+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee341a0410 con 0x7fee341006a0 2024-04-24T23:17:37.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.873+0000 7fee2affd640 1 -- 172.21.15.112:0/3138560684 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11970 (secure 0 0 0) 0x7fee2c09be40 con 0x7fee341006a0 2024-04-24T23:17:37.875 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:17:37.876 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:17:29.945643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24425,24424,24490,24482,24524]},"id":1}]} 2024-04-24T23:17:37.876 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 38 2024-04-24T23:17:37.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fee200a2b90 msgr2=0x7fee200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fee200a2b90 0x7fee200a5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee24005fd0 tx=0x7fee24005ea0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee341006a0 msgr2=0x7fee34197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee341006a0 0x7fee34197b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00e9c0 tx=0x7fee2c00ee80 comp rx=0 tx=0).stop 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 shutdown_connections 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fee200a2b90 0x7fee200a5050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fee34102250 0x7fee3419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee34101890 0x7fee34198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 --2- 172.21.15.112:0/3138560684 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee341006a0 0x7fee34197b50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 >> 172.21.15.112:0/3138560684 conn(0x7fee340fbc30 msgr2=0x7fee340fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 shutdown_connections 2024-04-24T23:17:37.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.875+0000 7fee3bda5640 1 -- 172.21.15.112:0/3138560684 wait complete. 2024-04-24T23:17:37.889 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-24T23:17:37.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 -- 172.21.15.112:0/1860577585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60101870 msgr2=0x7fac60101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 --2- 172.21.15.112:0/1860577585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60101870 0x7fac60101cf0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fac50009900 tx=0x7fac5002a410 comp rx=0 tx=0).stop 2024-04-24T23:17:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 -- 172.21.15.112:0/1860577585 shutdown_connections 2024-04-24T23:17:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 --2- 172.21.15.112:0/1860577585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac60102230 0x7fac60108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 --2- 172.21.15.112:0/1860577585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60101870 0x7fac60101cf0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 --2- 172.21.15.112:0/1860577585 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac60100680 0x7fac60100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 -- 172.21.15.112:0/1860577585 >> 172.21.15.112:0/1860577585 conn(0x7fac600fbbf0 msgr2=0x7fac600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 -- 172.21.15.112:0/1860577585 shutdown_connections 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.986+0000 7fac684e1640 1 -- 172.21.15.112:0/1860577585 wait complete. 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac684e1640 1 Processor -- start 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac684e1640 1 -- start start 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac684e1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac60100680 0x7fac60197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac684e1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac60101870 0x7fac60198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac684e1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60102230 0x7fac6019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac684e1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac6010c430 con 0x7fac60102230 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac684e1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac6010c2b0 con 0x7fac60100680 2024-04-24T23:17:37.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac684e1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac6010c5b0 con 0x7fac60101870 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac66a7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60102230 0x7fac6019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac66a7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60102230 0x7fac6019f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54618/0 (socket says 172.21.15.112:54618) 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac66a7e640 1 -- 172.21.15.112:0/9249888 learned_addr learned my addr 172.21.15.112:0/9249888 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac65a7c640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac60101870 0x7fac60198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.987+0000 7fac6627d640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac60100680 0x7fac60197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac66a7e640 1 -- 172.21.15.112:0/9249888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac60101870 msgr2=0x7fac60198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac66a7e640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac60101870 0x7fac60198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac66a7e640 1 -- 172.21.15.112:0/9249888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac60100680 msgr2=0x7fac60197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac66a7e640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac60100680 0x7fac60197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac66a7e640 1 -- 172.21.15.112:0/9249888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6019f7e0 con 0x7fac60102230 2024-04-24T23:17:37.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac66a7e640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60102230 0x7fac6019f170 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fac5800eaa0 tx=0x7fac5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac577fe640 1 -- 172.21.15.112:0/9249888 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5800cdb0 con 0x7fac60102230 2024-04-24T23:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac577fe640 1 -- 172.21.15.112:0/9249888 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fac58004b20 con 0x7fac60102230 2024-04-24T23:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac684e1640 1 -- 172.21.15.112:0/9249888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6019fad0 con 0x7fac60102230 2024-04-24T23:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac577fe640 1 -- 172.21.15.112:0/9249888 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58011760 con 0x7fac60102230 2024-04-24T23:17:37.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.988+0000 7fac684e1640 1 -- 172.21.15.112:0/9249888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac601a0040 con 0x7fac60102230 2024-04-24T23:17:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.990+0000 7fac577fe640 1 -- 172.21.15.112:0/9249888 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fac58004cd0 con 0x7fac60102230 2024-04-24T23:17:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.990+0000 7fac6b1f3740 1 Processor -- start 2024-04-24T23:17:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac6b1f3740 1 -- start start 2024-04-24T23:17:38.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653db9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dba030 0x558653dbeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dbf1a0 0x558653dc1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558653d19670 con 0x558653db9640 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558653cc1970 con 0x558653dbf1a0 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558653d15630 con 0x558653dba030 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac577fe640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fac4c0a2c10 0x7fac4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac65a7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653db9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac65a7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653db9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54622/0 (socket says 172.21.15.112:54622) 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac65a7c640 1 -- 172.21.15.112:0/2487707190 learned_addr learned my addr 172.21.15.112:0/2487707190 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac6627d640 1 --2- 172.21.15.112:0/2487707190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dba030 0x558653dbeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac66a7e640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fac4c0a2c10 0x7fac4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac577fe640 1 -- 172.21.15.112:0/9249888 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fac580cf6b0 con 0x7fac60102230 2024-04-24T23:17:38.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac65a7c640 1 -- 172.21.15.112:0/2487707190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dba030 msgr2=0x558653dbeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac65a7c640 1 --2- 172.21.15.112:0/2487707190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dba030 0x558653dbeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac65a7c640 1 -- 172.21.15.112:0/2487707190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dbf1a0 msgr2=0x558653dc1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac65a7c640 1 --2- 172.21.15.112:0/2487707190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dbf1a0 0x558653dc1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.991+0000 7fac65a7c640 1 -- 172.21.15.112:0/2487707190 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558653ce5400 con 0x558653db9640 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.992+0000 7fac66a7e640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fac4c0a2c10 0x7fac4c0a50d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac5809b820 tx=0x7fac5806db30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.992+0000 7fac65a7c640 1 --2- 172.21.15.112:0/2487707190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653db9a60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fac50005940 tx=0x7fac50009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4209ae3c0b237dd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.992+0000 7fac54bfe640 1 -- 172.21.15.112:0/2487707190 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50035050 con 0x558653db9640 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.992+0000 7fac54bfe640 1 -- 172.21.15.112:0/2487707190 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fac50031940 con 0x558653db9640 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.992+0000 7fac54bfe640 1 -- 172.21.15.112:0/2487707190 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50004920 con 0x558653db9640 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- 172.21.15.112:0/2487707190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 msgr2=0x558653db9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/2487707190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653db9a60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fac50005940 tx=0x7fac50009670 comp rx=0 tx=0).stop 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- 172.21.15.112:0/2487707190 shutdown_connections 2024-04-24T23:17:38.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/2487707190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dbf1a0 0x558653dc1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/2487707190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dba030 0x558653dbeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/2487707190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653db9a60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- 172.21.15.112:0/2487707190 >> 172.21.15.112:0/2487707190 conn(0x558653db08a0 msgr2=0x558653db0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- 172.21.15.112:0/2487707190 shutdown_connections 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- 172.21.15.112:0/2487707190 wait complete. 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- start start 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 Processor -- start 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653dc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 0x558653dc3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dbf1a0 0x558653dcace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558653dc20f0 con 0x558653db9640 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558653cd5050 con 0x558653dba030 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558653dc2270 con 0x558653dbf1a0 2024-04-24T23:17:38.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6b1f3740 10 client.0 _ll_register_callbacks cb 0x558653a2cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac65a7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653dc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac65a7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653dc3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54634/0 (socket says 172.21.15.112:54634) 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac65a7c640 1 -- 172.21.15.112:0/1727499109 learned_addr learned my addr 172.21.15.112:0/1727499109 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.993+0000 7fac6627d640 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 0x558653dc3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac65a7c640 1 -- 172.21.15.112:0/1727499109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dbf1a0 msgr2=0x558653dcace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac65a7c640 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dbf1a0 0x558653dcace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac65a7c640 1 -- 172.21.15.112:0/1727499109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 msgr2=0x558653dc3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac65a7c640 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 0x558653dc3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac65a7c640 1 -- 172.21.15.112:0/1727499109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558653ca6dc0 con 0x558653db9640 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac65a7c640 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653dc3760 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fac50005c40 tx=0x7fac50003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6435c8407d4087c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac49ffb640 1 -- 172.21.15.112:0/1727499109 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5004d070 con 0x558653db9640 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 1 -- 172.21.15.112:0/1727499109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac50031940 con 0x558653db9640 2024-04-24T23:17:38.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac49ffb640 1 -- 172.21.15.112:0/1727499109 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fac50003c20 con 0x558653db9640 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac49ffb640 1 -- 172.21.15.112:0/1727499109 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fac50035040 con 0x558653db9640 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 10 client.16320 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 1 -- 172.21.15.112:0/1727499109 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558653a2d3f0 con 0x558653db9640 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac49ffb640 1 -- 172.21.15.112:0/1727499109 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fac50031940 con 0x558653db9640 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac49ffb640 1 -- 172.21.15.112:0/1727499109 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7fac5003a410 con 0x558653db9640 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 10 client.16320 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 10 client.16320 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 20 client.16320 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558653dcf1d0 0x558653dd15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 4 client.16320 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.994+0000 7fac6b1f3740 1 -- 172.21.15.112:0/1727499109 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558653d26bb0 con 0x558653dcf1d0 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.995+0000 7fac6627d640 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558653dcf1d0 0x558653dd15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.995+0000 7fac6627d640 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558653dcf1d0 0x558653dd15c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c04c8347bd4edee8 server_cookie=2222ce282d8eb5e0 in_seq=0 out_seq=0 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.995+0000 7fac49ffb640 10 client.16320 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:17:38.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac49ffb640 1 -- 172.21.15.112:0/1727499109 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558653d26bb0 con 0x558653dcf1d0 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac49ffb640 10 client.16320 handle_command_reply: tid=0 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 client.16320 shutdown 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 -- 172.21.15.112:0/1727499109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 msgr2=0x558653dc3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653dc3760 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fac50005c40 tx=0x7fac50003820 comp rx=0 tx=0).stop 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 -- 172.21.15.112:0/1727499109 shutdown_connections 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558653dcf1d0 0x558653dd15c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653dbf1a0 0x558653dcace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 0x558653dc3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/1727499109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653db9640 0x558653dc3760 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 -- 172.21.15.112:0/1727499109 >> 172.21.15.112:0/1727499109 conn(0x558653db08a0 msgr2=0x558653dafb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac49ffb640 0 client.16320 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 -- 172.21.15.112:0/1727499109 shutdown_connections 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 1 -- 172.21.15.112:0/1727499109 wait complete. 2024-04-24T23:17:38.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:37.996+0000 7fac6b1f3740 20 client.16320 trim_cache size 0 max 16384 2024-04-24T23:17:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.009+0000 7fac6b1f3740 1 Processor -- start 2024-04-24T23:17:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.009+0000 7fac6b1f3740 1 -- start start 2024-04-24T23:17:38.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 0x558653dcac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dbf1a0 0x558653dbb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac601a83f0 con 0x558653dbf1a0 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac601aa110 con 0x558653dba030 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac601a8ac0 con 0x558653db9640 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dbf1a0 0x558653dbb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dbf1a0 0x558653dbb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54636/0 (socket says 172.21.15.112:54636) 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 -- 172.21.15.112:0/577894979 learned_addr learned my addr 172.21.15.112:0/577894979 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 --2- 172.21.15.112:0/577894979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 -- 172.21.15.112:0/577894979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 msgr2=0x558653dc3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 --2- 172.21.15.112:0/577894979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 -- 172.21.15.112:0/577894979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 msgr2=0x558653dcac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 --2- 172.21.15.112:0/577894979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 0x558653dcac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 -- 172.21.15.112:0/577894979 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558653af77d0 con 0x558653dbf1a0 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac65a7c640 1 --2- 172.21.15.112:0/577894979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dbf1a0 0x558653dbb1e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fac5003a7b0 tx=0x7fac50031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291173644d183454 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac4bfff640 1 -- 172.21.15.112:0/577894979 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5004d070 con 0x558653dbf1a0 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac4bfff640 1 -- 172.21.15.112:0/577894979 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fac50005570 con 0x558653dbf1a0 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.010+0000 7fac4bfff640 1 -- 172.21.15.112:0/577894979 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5003aa20 con 0x558653dbf1a0 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- 172.21.15.112:0/577894979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dbf1a0 msgr2=0x558653dbb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/577894979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dbf1a0 0x558653dbb1e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fac5003a7b0 tx=0x7fac50031660 comp rx=0 tx=0).stop 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- 172.21.15.112:0/577894979 shutdown_connections 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/577894979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dbf1a0 0x558653dbb1e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/577894979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dba030 0x558653dcac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/577894979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- 172.21.15.112:0/577894979 >> 172.21.15.112:0/577894979 conn(0x558653db08a0 msgr2=0x558653dbaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- 172.21.15.112:0/577894979 shutdown_connections 2024-04-24T23:17:38.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- 172.21.15.112:0/577894979 wait complete. 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- start start 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 Processor -- start 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dba030 0x558653dc8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dbf1a0 0x558653e33650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac601a9980 con 0x558653dba030 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fac601a9220 con 0x558653dbf1a0 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.011+0000 7fac6b1f3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac601a63a0 con 0x558653db9640 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6b1f3740 10 client.0 _ll_register_callbacks cb 0x7fac5c0092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dba030 0x558653dc8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dba030 0x558653dc8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54638/0 (socket says 172.21.15.112:54638) 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 -- 172.21.15.112:0/480418830 learned_addr learned my addr 172.21.15.112:0/480418830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac65a7c640 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 -- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 msgr2=0x558653dc7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 -- 172.21.15.112:0/480418830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dbf1a0 msgr2=0x558653e33650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dbf1a0 0x558653e33650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 -- 172.21.15.112:0/480418830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558653dc8610 con 0x558653dba030 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac65a7c640 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6627d640 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dba030 0x558653dc8020 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x558653dbaaa0 tx=0x7fac5c014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7bc1ba2dec479c server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac49ffb640 1 -- 172.21.15.112:0/480418830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c005220 con 0x558653dba030 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac49ffb640 1 -- 172.21.15.112:0/480418830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fac5c0053c0 con 0x558653dba030 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6b1f3740 1 -- 172.21.15.112:0/480418830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac50005570 con 0x558653dba030 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac49ffb640 1 -- 172.21.15.112:0/480418830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fac5c02c050 con 0x558653dba030 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6b1f3740 10 client.16326 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac6b1f3740 1 -- 172.21.15.112:0/480418830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558653e34d00 con 0x558653dba030 2024-04-24T23:17:38.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.012+0000 7fac49ffb640 1 -- 172.21.15.112:0/480418830 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fac50005570 con 0x558653dba030 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac49ffb640 1 -- 172.21.15.112:0/480418830 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7fac5c00f040 con 0x558653dba030 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac6b1f3740 10 client.16326 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac6b1f3740 10 client.16326 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac6b1f3740 20 client.16326 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558653e357a0 0x558653e37bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac6b1f3740 4 client.16326 mds_command: new command op to 24424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac6b1f3740 1 -- 172.21.15.112:0/480418830 --> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558653d20f00 con 0x558653e357a0 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac65a7c640 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558653e357a0 0x558653e37bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac49ffb640 10 client.16326 ms_handle_connect on v2:172.21.15.187:6832/1998066775 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac65a7c640 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558653e357a0 0x558653e37bc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c58db18a9cd28caa server_cookie=eb1e633403900c57 in_seq=0 out_seq=0 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac49ffb640 1 -- 172.21.15.112:0/480418830 <== mds.4 v2:172.21.15.187:6832/1998066775 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558653d20f00 con 0x558653e357a0 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac49ffb640 10 client.16326 handle_command_reply: tid=0 2024-04-24T23:17:38.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.013+0000 7fac6b1f3740 1 client.16326 shutdown 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 -- 172.21.15.112:0/480418830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dba030 msgr2=0x558653dc8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dba030 0x558653dc8020 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x558653dbaaa0 tx=0x7fac5c014840 comp rx=0 tx=0).stop 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 -- 172.21.15.112:0/480418830 shutdown_connections 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:6832/1998066775,v1:172.21.15.187:6833/1998066775] conn(0x558653e357a0 0x558653e37bc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558653dbf1a0 0x558653e33650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac49ffb640 0 client.16326 ms_handle_reset on v2:172.21.15.187:6832/1998066775 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558653dba030 0x558653dc8020 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 --2- 172.21.15.112:0/480418830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558653db9640 0x558653dc7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 -- 172.21.15.112:0/480418830 >> 172.21.15.112:0/480418830 conn(0x558653db08a0 msgr2=0x558653dc3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 -- 172.21.15.112:0/480418830 shutdown_connections 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 1 -- 172.21.15.112:0/480418830 wait complete. 2024-04-24T23:17:38.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac6b1f3740 20 client.16326 trim_cache size 0 max 16384 2024-04-24T23:17:38.024 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:38.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 -- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fac4c0a2c10 msgr2=0x7fac4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fac4c0a2c10 0x7fac4c0a50d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fac5809b820 tx=0x7fac5806db30 comp rx=0 tx=0).stop 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 -- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60102230 msgr2=0x7fac6019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60102230 0x7fac6019f170 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fac5800eaa0 tx=0x7fac5800ef60 comp rx=0 tx=0).stop 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 -- 172.21.15.112:0/9249888 shutdown_connections 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fac4c0a2c10 0x7fac4c0a50d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac60102230 0x7fac6019f170 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac60101870 0x7fac60198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 --2- 172.21.15.112:0/9249888 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fac60100680 0x7fac60197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 -- 172.21.15.112:0/9249888 >> 172.21.15.112:0/9249888 conn(0x7fac600fbbf0 msgr2=0x7fac600fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 -- 172.21.15.112:0/9249888 shutdown_connections 2024-04-24T23:17:38.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.014+0000 7fac684e1640 1 -- 172.21.15.112:0/9249888 wait complete. 2024-04-24T23:17:38.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-24T23:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 -- 172.21.15.112:0/2965574271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c103330 msgr2=0x7ff64c109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2965574271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c103330 0x7ff64c109bc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff644009ca0 tx=0x7ff64402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 -- 172.21.15.112:0/2965574271 shutdown_connections 2024-04-24T23:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2965574271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c103330 0x7ff64c109bc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2965574271 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c102970 0x7ff64c102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2965574271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff64c101780 0x7ff64c101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 -- 172.21.15.112:0/2965574271 >> 172.21.15.112:0/2965574271 conn(0x7ff64c0fccf0 msgr2=0x7ff64c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 -- 172.21.15.112:0/2965574271 shutdown_connections 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.125+0000 7ff6512a5640 1 -- 172.21.15.112:0/2965574271 wait complete. 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff6512a5640 1 Processor -- start 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff6512a5640 1 -- start start 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff6512a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c101780 0x7ff64c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff6512a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c102970 0x7ff64c198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff6512a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff64c103330 0x7ff64c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff6512a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10d530 con 0x7ff64c102970 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff6512a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10d3b0 con 0x7ff64c101780 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff6512a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10d6b0 con 0x7ff64c103330 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff64a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c102970 0x7ff64c198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.126+0000 7ff64a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c102970 0x7ff64c198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54650/0 (socket says 172.21.15.112:54650) 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64a7fc640 1 -- 172.21.15.112:0/2573063260 learned_addr learned my addr 172.21.15.112:0/2573063260 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff64c103330 0x7ff64c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64affd640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c101780 0x7ff64c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64a7fc640 1 -- 172.21.15.112:0/2573063260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff64c103330 msgr2=0x7ff64c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff64c103330 0x7ff64c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64a7fc640 1 -- 172.21.15.112:0/2573063260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c101780 msgr2=0x7ff64c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c101780 0x7ff64c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64a7fc640 1 -- 172.21.15.112:0/2573063260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c19f6e0 con 0x7ff64c102970 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c102970 0x7ff64c198000 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c920 tx=0x7ff64000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff633fff640 1 -- 172.21.15.112:0/2573063260 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640007c70 con 0x7ff64c102970 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff6512a5640 1 -- 172.21.15.112:0/2573063260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c19f9d0 con 0x7ff64c102970 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff633fff640 1 -- 172.21.15.112:0/2573063260 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff640007e10 con 0x7ff64c102970 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff633fff640 1 -- 172.21.15.112:0/2573063260 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640018940 con 0x7ff64c102970 2024-04-24T23:17:38.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.127+0000 7ff6512a5640 1 -- 172.21.15.112:0/2573063260 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c19ff20 con 0x7ff64c102970 2024-04-24T23:17:38.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.129+0000 7ff633fff640 1 -- 172.21.15.112:0/2573063260 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff640018b20 con 0x7ff64c102970 2024-04-24T23:17:38.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.129+0000 7ff653fb8740 1 Processor -- start 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff653fb8740 1 -- start start 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x558132378650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x55813237d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x558132380220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff653fb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581322dc670 con 0x55813237dd90 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff653fb8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558132284970 con 0x558132378c20 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff653fb8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5581322d8630 con 0x558132378230 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x558132380220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x558132380220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54660/0 (socket says 172.21.15.112:54660) 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 -- 172.21.15.112:0/3015179100 learned_addr learned my addr 172.21.15.112:0/3015179100 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff633fff640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6340a2b90 0x7ff6340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x558132378650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6340a2b90 0x7ff6340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64affd640 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x55813237d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 -- 172.21.15.112:0/3015179100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 msgr2=0x558132378650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x558132378650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 -- 172.21.15.112:0/3015179100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 msgr2=0x55813237d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x55813237d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff633fff640 1 -- 172.21.15.112:0/2573063260 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff6400cf500 con 0x7ff64c102970 2024-04-24T23:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 -- 172.21.15.112:0/3015179100 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581322a8400 con 0x55813237dd90 2024-04-24T23:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64affd640 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x55813237d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x558132378650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x558132380220 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff64009b900 tx=0x7ff640000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad4376659b1d44d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.130+0000 7ff6317fa640 1 -- 172.21.15.112:0/3015179100 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6400cf020 con 0x55813237dd90 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff6317fa640 1 -- 172.21.15.112:0/3015179100 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff64009c070 con 0x55813237dd90 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff6317fa640 1 -- 172.21.15.112:0/3015179100 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640097fe0 con 0x55813237dd90 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6340a2b90 0x7ff6340a5050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558132379480 tx=0x7ff6440095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 -- 172.21.15.112:0/3015179100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 msgr2=0x558132380220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x558132380220 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff64009b900 tx=0x7ff640000bc0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 -- 172.21.15.112:0/3015179100 shutdown_connections 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x558132380220 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x55813237d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3015179100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x558132378650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 -- 172.21.15.112:0/3015179100 >> 172.21.15.112:0/3015179100 conn(0x5581323738a0 msgr2=0x558132373ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 -- 172.21.15.112:0/3015179100 shutdown_connections 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 -- 172.21.15.112:0/3015179100 wait complete. 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.131+0000 7ff653fb8740 1 -- start start 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 1 Processor -- start 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x5581323822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558132378c20 0x558132382830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55813237dd90 0x558132389870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558132380ce0 con 0x558132378c20 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x558132298050 con 0x55813237dd90 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558132380e60 con 0x558132378230 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 10 client.0 _ll_register_callbacks cb 0x558131fefbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558132378c20 0x558132382830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558132378c20 0x558132382830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54668/0 (socket says 172.21.15.112:54668) 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 -- 172.21.15.112:0/138328756 learned_addr learned my addr 172.21.15.112:0/138328756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64affd640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x5581323822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55813237dd90 0x558132389870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 -- 172.21.15.112:0/138328756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 msgr2=0x5581323822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x5581323822f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 -- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55813237dd90 msgr2=0x558132389870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55813237dd90 0x558132389870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 -- 172.21.15.112:0/138328756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558132269dc0 con 0x558132378c20 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55813237dd90 0x558132389870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64affd640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x5581323822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558132378c20 0x558132382830 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c8f0 tx=0x7ff64009fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4342da62271491eb server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff62eff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff62eff5640 1 -- 172.21.15.112:0/138328756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6400cf070 con 0x558132378c20 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff62eff5640 1 -- 172.21.15.112:0/138328756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff6400976c0 con 0x558132378c20 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.132+0000 7ff653fb8740 1 -- 172.21.15.112:0/138328756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64009c070 con 0x558132378c20 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff62eff5640 1 -- 172.21.15.112:0/138328756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff64004d070 con 0x558132378c20 2024-04-24T23:17:38.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff653fb8740 10 client.16338 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff653fb8740 1 -- 172.21.15.112:0/138328756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558131ff03f0 con 0x558132378c20 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff62eff5640 1 -- 172.21.15.112:0/138328756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff64009c070 con 0x558132378c20 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff62eff5640 1 -- 172.21.15.112:0/138328756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7ff6400861f0 con 0x558132378c20 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff653fb8740 10 client.16338 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff653fb8740 10 client.16338 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff653fb8740 20 client.16338 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff653fb8740 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55813238dcd0 0x5581323900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff653fb8740 4 client.16338 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff653fb8740 1 -- 172.21.15.112:0/138328756 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581322e9bb0 con 0x55813238dcd0 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.133+0000 7ff64affd640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55813238dcd0 0x5581323900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.134+0000 7ff64affd640 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55813238dcd0 0x5581323900c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb48b810b93969bf server_cookie=6d88f4cb3edc5c47 in_seq=0 out_seq=0 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.134+0000 7ff62eff5640 10 client.16338 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:17:38.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff62eff5640 1 -- 172.21.15.112:0/138328756 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581322e9bb0 con 0x55813238dcd0 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff62eff5640 10 client.16338 handle_command_reply: tid=0 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 client.16338 shutdown 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 -- 172.21.15.112:0/138328756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558132378c20 msgr2=0x558132382830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558132378c20 0x558132382830 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c8f0 tx=0x7ff64009fd20 comp rx=0 tx=0).stop 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 -- 172.21.15.112:0/138328756 shutdown_connections 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x55813238dcd0 0x5581323900c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55813237dd90 0x558132389870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff62eff5640 0 client.16338 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558132378c20 0x558132382830 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 --2- 172.21.15.112:0/138328756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x5581323822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 -- 172.21.15.112:0/138328756 >> 172.21.15.112:0/138328756 conn(0x5581323738a0 msgr2=0x558132372b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 -- 172.21.15.112:0/138328756 shutdown_connections 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 1 -- 172.21.15.112:0/138328756 wait complete. 2024-04-24T23:17:38.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.135+0000 7ff653fb8740 20 client.16338 trim_cache size 0 max 16384 2024-04-24T23:17:38.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff653fb8740 1 Processor -- start 2024-04-24T23:17:38.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff653fb8740 1 -- start start 2024-04-24T23:17:38.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x55813237d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x558132389830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff653fb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1aa5d0 con 0x55813237dd90 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff653fb8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1ac2f0 con 0x558132378c20 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff653fb8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1aaca0 con 0x558132378230 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54678/0 (socket says 172.21.15.112:54678) 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64b7fe640 1 -- 172.21.15.112:0/3677750879 learned_addr learned my addr 172.21.15.112:0/3677750879 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64affd640 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x55813237d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64b7fe640 1 -- 172.21.15.112:0/3677750879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 msgr2=0x55813237d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x558132389830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x55813237d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64b7fe640 1 -- 172.21.15.112:0/3677750879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 msgr2=0x558132389830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x558132389830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64b7fe640 1 -- 172.21.15.112:0/3677750879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581320ba7d0 con 0x55813237dd90 2024-04-24T23:17:38.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.148+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x558132389830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323767f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff644035000 tx=0x7ff644043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b4804279d2d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff630ff9640 1 -- 172.21.15.112:0/3677750879 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644031770 con 0x55813237dd90 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff630ff9640 1 -- 172.21.15.112:0/3677750879 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff644036040 con 0x55813237dd90 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff630ff9640 1 -- 172.21.15.112:0/3677750879 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6440305c0 con 0x55813237dd90 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 -- 172.21.15.112:0/3677750879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 msgr2=0x5581323767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323767f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff644035000 tx=0x7ff644043000 comp rx=0 tx=0).stop 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 -- 172.21.15.112:0/3677750879 shutdown_connections 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323767f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378c20 0x558132389830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3677750879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378230 0x55813237d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 -- 172.21.15.112:0/3677750879 >> 172.21.15.112:0/3677750879 conn(0x5581323738a0 msgr2=0x558132372b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 -- 172.21.15.112:0/3677750879 shutdown_connections 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.149+0000 7ff653fb8740 1 -- 172.21.15.112:0/3677750879 wait complete. 2024-04-24T23:17:38.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 1 -- start start 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 1 Processor -- start 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378230 0x558132387380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378c20 0x5581323878c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1abb60 con 0x55813237dd90 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1ab400 con 0x558132378230 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1a8580 con 0x558132378c20 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff653fb8740 10 client.0 _ll_register_callbacks cb 0x7ff64009b4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323888d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54680/0 (socket says 172.21.15.112:54680) 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 -- 172.21.15.112:0/3319541199 learned_addr learned my addr 172.21.15.112:0/3319541199 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64a7fc640 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378c20 0x5581323878c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64affd640 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378230 0x558132387380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 -- 172.21.15.112:0/3319541199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378c20 msgr2=0x5581323878c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378c20 0x5581323878c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 -- 172.21.15.112:0/3319541199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378230 msgr2=0x558132387380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378230 0x558132387380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 -- 172.21.15.112:0/3319541199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558132388ec0 con 0x55813237dd90 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff64b7fe640 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323888d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff644009ca0 tx=0x7ff644005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776054374cf2813d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.150+0000 7ff62eff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff62eff5640 1 -- 172.21.15.112:0/3319541199 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64405c060 con 0x55813237dd90 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff62eff5640 1 -- 172.21.15.112:0/3319541199 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7ff644031770 con 0x55813237dd90 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 1 -- 172.21.15.112:0/3319541199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff644036040 con 0x55813237dd90 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff62eff5640 1 -- 172.21.15.112:0/3319541199 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff644053050 con 0x55813237dd90 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 10 client.16344 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 1 -- 172.21.15.112:0/3319541199 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5581323f45e0 con 0x55813237dd90 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff62eff5640 1 -- 172.21.15.112:0/3319541199 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff644036040 con 0x55813237dd90 2024-04-24T23:17:38.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff62eff5640 1 -- 172.21.15.112:0/3319541199 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7ff644049550 con 0x55813237dd90 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 10 client.16344 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 10 client.16344 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 20 client.16344 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5581323f4fa0 0x5581323f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 4 client.16344 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff653fb8740 1 -- 172.21.15.112:0/3319541199 --> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581322e3f00 con 0x5581323f4fa0 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.151+0000 7ff64affd640 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5581323f4fa0 0x5581323f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff62eff5640 10 client.16344 ms_handle_connect on v2:172.21.15.150:6832/68455606 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff64affd640 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5581323f4fa0 0x5581323f73c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95d4fc62ccba02da server_cookie=ff3bf97c7804ab6e in_seq=0 out_seq=0 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff62eff5640 1 -- 172.21.15.112:0/3319541199 <== mds.3 v2:172.21.15.150:6832/68455606 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581322e3f00 con 0x5581323f4fa0 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff62eff5640 10 client.16344 handle_command_reply: tid=0 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 client.16344 shutdown 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 -- 172.21.15.112:0/3319541199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 msgr2=0x5581323888d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323888d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff644009ca0 tx=0x7ff644005f30 comp rx=0 tx=0).stop 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 -- 172.21.15.112:0/3319541199 shutdown_connections 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.150:6832/68455606,v1:172.21.15.150:6833/68455606] conn(0x5581323f4fa0 0x5581323f73c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55813237dd90 0x5581323888d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558132378c20 0x5581323878c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff62eff5640 0 client.16344 ms_handle_reset on v2:172.21.15.150:6832/68455606 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 --2- 172.21.15.112:0/3319541199 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558132378230 0x558132387380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 -- 172.21.15.112:0/3319541199 >> 172.21.15.112:0/3319541199 conn(0x5581323738a0 msgr2=0x5581323722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 -- 172.21.15.112:0/3319541199 shutdown_connections 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 1 -- 172.21.15.112:0/3319541199 wait complete. 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.152+0000 7ff653fb8740 20 client.16344 trim_cache size 0 max 16384 2024-04-24T23:17:38.161 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 -- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6340a2b90 msgr2=0x7ff6340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6340a2b90 0x7ff6340a5050 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x558132379480 tx=0x7ff6440095c0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 -- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c102970 msgr2=0x7ff64c198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c102970 0x7ff64c198000 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff64000c920 tx=0x7ff64000cde0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 -- 172.21.15.112:0/2573063260 shutdown_connections 2024-04-24T23:17:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff6340a2b90 0x7ff6340a5050 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff64c103330 0x7ff64c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff64c102970 0x7ff64c198000 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 --2- 172.21.15.112:0/2573063260 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c101780 0x7ff64c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 -- 172.21.15.112:0/2573063260 >> 172.21.15.112:0/2573063260 conn(0x7ff64c0fccf0 msgr2=0x7ff64c105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 -- 172.21.15.112:0/2573063260 shutdown_connections 2024-04-24T23:17:38.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.153+0000 7ff6512a5640 1 -- 172.21.15.112:0/2573063260 wait complete. 2024-04-24T23:17:38.164 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:38 smithi112 ceph-mon[29186]: pgmap v449: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 475 KiB/s wr, 144 op/s 2024-04-24T23:17:38.164 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:38 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3138560684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:38.166 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-24T23:17:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 -- 172.21.15.112:0/1721974101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c102250 msgr2=0x7f216c108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 --2- 172.21.15.112:0/1721974101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c102250 0x7f216c108ae0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2164009ca0 tx=0x7f216402a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 -- 172.21.15.112:0/1721974101 shutdown_connections 2024-04-24T23:17:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 --2- 172.21.15.112:0/1721974101 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c102250 0x7f216c108ae0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 --2- 172.21.15.112:0/1721974101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f216c101890 0x7f216c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 --2- 172.21.15.112:0/1721974101 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c1006a0 0x7f216c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 -- 172.21.15.112:0/1721974101 >> 172.21.15.112:0/1721974101 conn(0x7f216c0fbc30 msgr2=0x7f216c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 -- 172.21.15.112:0/1721974101 shutdown_connections 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 -- 172.21.15.112:0/1721974101 wait complete. 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 Processor -- start 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.264+0000 7f2173eb3640 1 -- start start 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2173eb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f216c1006a0 0x7f216c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2173eb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c101890 0x7f216c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2173eb3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c102250 0x7f216c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2173eb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f216c10d560 con 0x7f216c101890 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2173eb3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f216c10d3e0 con 0x7f216c102250 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2173eb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f216c10d6e0 con 0x7f216c1006a0 2024-04-24T23:17:38.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c101890 0x7f216c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c101890 0x7f216c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54694/0 (socket says 172.21.15.112:54694) 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 -- 172.21.15.112:0/4002857456 learned_addr learned my addr 172.21.15.112:0/4002857456 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2171c4f640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f216c1006a0 0x7f216c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2172450640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c102250 0x7f216c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 -- 172.21.15.112:0/4002857456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f216c1006a0 msgr2=0x7f216c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f216c1006a0 0x7f216c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 -- 172.21.15.112:0/4002857456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c102250 msgr2=0x7f216c19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c102250 0x7f216c19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 -- 172.21.15.112:0/4002857456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c19f760 con 0x7f216c101890 2024-04-24T23:17:38.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2172450640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c102250 0x7f216c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2171c4f640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f216c1006a0 0x7f216c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f217144e640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c101890 0x7f216c198080 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f215c00e9f0 tx=0x7f215c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2162ffd640 1 -- 172.21.15.112:0/4002857456 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c00cd80 con 0x7f216c101890 2024-04-24T23:17:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2162ffd640 1 -- 172.21.15.112:0/4002857456 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f215c005520 con 0x7f216c101890 2024-04-24T23:17:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2173eb3640 1 -- 172.21.15.112:0/4002857456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216c19fa50 con 0x7f216c101890 2024-04-24T23:17:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2162ffd640 1 -- 172.21.15.112:0/4002857456 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c010690 con 0x7f216c101890 2024-04-24T23:17:38.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.265+0000 7f2173eb3640 1 -- 172.21.15.112:0/4002857456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216c19ff50 con 0x7f216c101890 2024-04-24T23:17:38.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.267+0000 7f2162ffd640 1 -- 172.21.15.112:0/4002857456 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f215c0046c0 con 0x7f216c101890 2024-04-24T23:17:38.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2176bbf740 1 Processor -- start 2024-04-24T23:17:38.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2176bbf740 1 -- start start 2024-04-24T23:17:38.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8982ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89831b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f89834630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2176bbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557f8978c670 con 0x557f8982d030 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2176bbf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557f89734970 con 0x557f8982c640 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2176bbf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557f89788630 con 0x557f898321a0 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2171c4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89831b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2171c4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89831b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54698/0 (socket says 172.21.15.112:54698) 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2171c4f640 1 -- 172.21.15.112:0/1411728278 learned_addr learned my addr 172.21.15.112:0/1411728278 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2162ffd640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f21580a2b90 0x7f21580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2172450640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f21580a2b90 0x7f21580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.268+0000 7f2172450640 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8982ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2162ffd640 1 -- 172.21.15.112:0/4002857456 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f215c0cf990 con 0x7f216c101890 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f217144e640 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f89834630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2171c4f640 1 -- 172.21.15.112:0/1411728278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 msgr2=0x557f89834630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2171c4f640 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f89834630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2171c4f640 1 -- 172.21.15.112:0/1411728278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 msgr2=0x557f8982ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2171c4f640 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8982ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2171c4f640 1 -- 172.21.15.112:0/1411728278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f89758400 con 0x557f8982d030 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2172450640 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8982ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f217144e640 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f89834630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2171c4f640 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89831b30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f216800cc50 tx=0x7f216800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95c2383dad22f31 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2172450640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f21580a2b90 0x7f21580a5050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x557f8982d890 tx=0x7f21640095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2157fff640 1 -- 172.21.15.112:0/1411728278 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168014070 con 0x557f8982d030 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2157fff640 1 -- 172.21.15.112:0/1411728278 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f21680047a0 con 0x557f8982d030 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.269+0000 7f2157fff640 1 -- 172.21.15.112:0/1411728278 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21680107d0 con 0x557f8982d030 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 -- 172.21.15.112:0/1411728278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 msgr2=0x557f89831b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89831b30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f216800cc50 tx=0x7f216800e720 comp rx=0 tx=0).stop 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 -- 172.21.15.112:0/1411728278 shutdown_connections 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f89834630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89831b30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 --2- 172.21.15.112:0/1411728278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8982ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 -- 172.21.15.112:0/1411728278 >> 172.21.15.112:0/1411728278 conn(0x557f898238a0 msgr2=0x557f89823ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 -- 172.21.15.112:0/1411728278 shutdown_connections 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 -- 172.21.15.112:0/1411728278 wait complete. 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 -- start start 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 Processor -- start 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f89836700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.270+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89836c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f8983dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2176bbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557f89835150 con 0x557f8982d030 2024-04-24T23:17:38.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2176bbf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x557f89748050 con 0x557f8982c640 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2176bbf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557f898352d0 con 0x557f898321a0 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2176bbf740 10 client.0 _ll_register_callbacks cb 0x557f8949fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89836c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89836c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54710/0 (socket says 172.21.15.112:54710) 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 -- 172.21.15.112:0/685104112 learned_addr learned my addr 172.21.15.112:0/685104112 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2171c4f640 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f89836700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 -- 172.21.15.112:0/685104112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 msgr2=0x557f8983dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2172450640 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f8983dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f8983dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 -- 172.21.15.112:0/685104112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 msgr2=0x557f89836700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f89836700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 -- 172.21.15.112:0/685104112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f89719dc0 con 0x557f8982d030 2024-04-24T23:17:38.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2172450640 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f8983dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f217144e640 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89836c40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f215c0cd0a0 tx=0x7f215c09b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0cd75019e5d1bd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f21557fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f21557fa640 1 -- 172.21.15.112:0/685104112 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c0a4a70 con 0x557f8982d030 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f21557fa640 1 -- 172.21.15.112:0/685104112 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f215c016db0 con 0x557f8982d030 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2176bbf740 1 -- 172.21.15.112:0/685104112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21680047a0 con 0x557f8982d030 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f21557fa640 1 -- 172.21.15.112:0/685104112 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f215c0a45e0 con 0x557f8982d030 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2176bbf740 10 client.16356 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.271+0000 7f2176bbf740 1 -- 172.21.15.112:0/685104112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557f894a03f0 con 0x557f8982d030 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f21557fa640 1 -- 172.21.15.112:0/685104112 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f21680047a0 con 0x557f8982d030 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f21557fa640 1 -- 172.21.15.112:0/685104112 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7f215c097b50 con 0x557f8982d030 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f2176bbf740 10 client.16356 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f2176bbf740 10 client.16356 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f2176bbf740 20 client.16356 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f2176bbf740 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x557f898425b0 0x557f89844a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f2176bbf740 4 client.16356 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f2176bbf740 1 -- 172.21.15.112:0/685104112 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f89799bb0 con 0x557f898425b0 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.272+0000 7f2171c4f640 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x557f898425b0 0x557f89844a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f21557fa640 10 client.16356 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2171c4f640 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x557f898425b0 0x557f89844a30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deea4957d344b095 server_cookie=fccfb347dccb8045 in_seq=0 out_seq=0 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f21557fa640 1 -- 172.21.15.112:0/685104112 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f89799bb0 con 0x557f898425b0 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f21557fa640 10 client.16356 handle_command_reply: tid=0 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 client.16356 shutdown 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 -- 172.21.15.112:0/685104112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 msgr2=0x557f89836c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89836c40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f215c0cd0a0 tx=0x7f215c09b820 comp rx=0 tx=0).stop 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 -- 172.21.15.112:0/685104112 shutdown_connections 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x557f898425b0 0x557f89844a30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f898321a0 0x557f8983dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f89836c40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f21557fa640 0 client.16356 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:17:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 --2- 172.21.15.112:0/685104112 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f89836700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.273+0000 7f2176bbf740 1 -- 172.21.15.112:0/685104112 >> 172.21.15.112:0/685104112 conn(0x557f898238a0 msgr2=0x557f89822b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.274+0000 7f2176bbf740 1 -- 172.21.15.112:0/685104112 shutdown_connections 2024-04-24T23:17:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.274+0000 7f2176bbf740 1 -- 172.21.15.112:0/685104112 wait complete. 2024-04-24T23:17:38.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.274+0000 7f2176bbf740 20 client.16356 trim_cache size 0 max 16384 2024-04-24T23:17:38.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f2176bbf740 1 Processor -- start 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f2176bbf740 1 -- start start 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982c640 0x557f8982dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f8983dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f898321a0 0x557f8982e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f2176bbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f216c1a8470 con 0x557f8982d030 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f2176bbf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f216c1aa190 con 0x557f898321a0 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f2176bbf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f216c1a8b40 con 0x557f8982c640 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f8983dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f8983dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54720/0 (socket says 172.21.15.112:54720) 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 -- 172.21.15.112:0/2813927448 learned_addr learned my addr 172.21.15.112:0/2813927448 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 -- 172.21.15.112:0/2813927448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982c640 msgr2=0x557f8982dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 --2- 172.21.15.112:0/2813927448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982c640 0x557f8982dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 -- 172.21.15.112:0/2813927448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f898321a0 msgr2=0x557f8982e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 --2- 172.21.15.112:0/2813927448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f898321a0 0x557f8982e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 -- 172.21.15.112:0/2813927448 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f89831ef0 con 0x557f8982d030 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f217144e640 1 --2- 172.21.15.112:0/2813927448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f8983dc40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f215c098260 tx=0x7f215c09a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3dcd0693c838ac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f21577fe640 1 -- 172.21.15.112:0/2813927448 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c0a4a70 con 0x557f8982d030 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f21577fe640 1 -- 172.21.15.112:0/2813927448 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f215c09acc0 con 0x557f8982d030 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.287+0000 7f21577fe640 1 -- 172.21.15.112:0/2813927448 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c09aef0 con 0x557f8982d030 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 -- 172.21.15.112:0/2813927448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 msgr2=0x557f8983dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 --2- 172.21.15.112:0/2813927448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f8983dc40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f215c098260 tx=0x7f215c09a310 comp rx=0 tx=0).stop 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 -- 172.21.15.112:0/2813927448 shutdown_connections 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 --2- 172.21.15.112:0/2813927448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f898321a0 0x557f8982e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 --2- 172.21.15.112:0/2813927448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f8982d030 0x557f8983dc40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 --2- 172.21.15.112:0/2813927448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982c640 0x557f8982dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 -- 172.21.15.112:0/2813927448 >> 172.21.15.112:0/2813927448 conn(0x557f898238a0 msgr2=0x557f89822b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 -- 172.21.15.112:0/2813927448 shutdown_connections 2024-04-24T23:17:38.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 -- 172.21.15.112:0/2813927448 wait complete. 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 -- start start 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.288+0000 7f2176bbf740 1 Processor -- start 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8983b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982d030 0x557f8983be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2176bbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f898321a0 0x557f8983ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2176bbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f216c1a9a00 con 0x557f898321a0 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2176bbf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f216c1a92a0 con 0x557f8982c640 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2176bbf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f216c1a6420 con 0x557f8982d030 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2176bbf740 10 client.0 _ll_register_callbacks cb 0x7f215c004b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f898321a0 0x557f8983ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f898321a0 0x557f8983ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54736/0 (socket says 172.21.15.112:54736) 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 -- 172.21.15.112:0/957467775 learned_addr learned my addr 172.21.15.112:0/957467775 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f217144e640 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982d030 0x557f8983be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2171c4f640 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8983b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 -- 172.21.15.112:0/957467775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982d030 msgr2=0x557f8983be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982d030 0x557f8983be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 -- 172.21.15.112:0/957467775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 msgr2=0x557f8983b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8983b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 -- 172.21.15.112:0/957467775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f8956a7d0 con 0x557f898321a0 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f217144e640 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982d030 0x557f8983be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2172450640 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f898321a0 0x557f8983ce90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2164031950 tx=0x7f2164005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeb5fa6cbffe382 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f21557fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f21557fa640 1 -- 172.21.15.112:0/957467775 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2164004250 con 0x557f898321a0 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f21557fa640 1 -- 172.21.15.112:0/957467775 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f21640043f0 con 0x557f898321a0 2024-04-24T23:17:38.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.289+0000 7f2176bbf740 1 -- 172.21.15.112:0/957467775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f215c09acc0 con 0x557f898321a0 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f21557fa640 1 -- 172.21.15.112:0/957467775 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f216405a050 con 0x557f898321a0 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2176bbf740 10 client.16362 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2176bbf740 1 -- 172.21.15.112:0/957467775 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557f898a9050 con 0x557f898321a0 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f21557fa640 1 -- 172.21.15.112:0/957467775 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f215c09acc0 con 0x557f898321a0 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f21557fa640 1 -- 172.21.15.112:0/957467775 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7f216404f020 con 0x557f898321a0 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2176bbf740 10 client.16362 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2176bbf740 10 client.16362 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2176bbf740 20 client.16362 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2176bbf740 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x557f898a9b40 0x557f898abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2176bbf740 4 client.16362 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2176bbf740 1 -- 172.21.15.112:0/957467775 --> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f89793f00 con 0x557f898a9b40 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.290+0000 7f2171c4f640 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x557f898a9b40 0x557f898abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f21557fa640 10 client.16362 ms_handle_connect on v2:172.21.15.150:6836/3592489674 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2171c4f640 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x557f898a9b40 0x557f898abf60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7cd42ceea88aaaf server_cookie=d3f41b7caea41163 in_seq=0 out_seq=0 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f21557fa640 1 -- 172.21.15.112:0/957467775 <== mds.2 v2:172.21.15.150:6836/3592489674 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f89793f00 con 0x557f898a9b40 2024-04-24T23:17:38.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f21557fa640 10 client.16362 handle_command_reply: tid=0 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 client.16362 shutdown 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 -- 172.21.15.112:0/957467775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f898321a0 msgr2=0x557f8983ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f898321a0 0x557f8983ce90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2164031950 tx=0x7f2164005d80 comp rx=0 tx=0).stop 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 -- 172.21.15.112:0/957467775 shutdown_connections 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.150:6836/3592489674,v1:172.21.15.150:6837/3592489674] conn(0x557f898a9b40 0x557f898abf60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557f898321a0 0x557f8983ce90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f21557fa640 0 client.16362 ms_handle_reset on v2:172.21.15.150:6836/3592489674 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f8982d030 0x557f8983be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 --2- 172.21.15.112:0/957467775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557f8982c640 0x557f8983b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 -- 172.21.15.112:0/957467775 >> 172.21.15.112:0/957467775 conn(0x557f898238a0 msgr2=0x557f89826890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 -- 172.21.15.112:0/957467775 shutdown_connections 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.291+0000 7f2176bbf740 1 -- 172.21.15.112:0/957467775 wait complete. 2024-04-24T23:17:38.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2176bbf740 20 client.16362 trim_cache size 0 max 16384 2024-04-24T23:17:38.302 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:38.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 -- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f21580a2b90 msgr2=0x7f21580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f21580a2b90 0x7f21580a5050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x557f8982d890 tx=0x7f21640095c0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 -- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c101890 msgr2=0x7f216c198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c101890 0x7f216c198080 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f215c00e9f0 tx=0x7f215c00edf0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 -- 172.21.15.112:0/4002857456 shutdown_connections 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f21580a2b90 0x7f21580a5050 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c102250 0x7f216c19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216c101890 0x7f216c198080 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 --2- 172.21.15.112:0/4002857456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f216c1006a0 0x7f216c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 -- 172.21.15.112:0/4002857456 >> 172.21.15.112:0/4002857456 conn(0x7f216c0fbc30 msgr2=0x7f216c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 -- 172.21.15.112:0/4002857456 shutdown_connections 2024-04-24T23:17:38.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.292+0000 7f2173eb3640 1 -- 172.21.15.112:0/4002857456 wait complete. 2024-04-24T23:17:38.305 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-24T23:17:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:38 smithi150 ceph-mon[32247]: pgmap v449: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 475 KiB/s wr, 144 op/s 2024-04-24T23:17:38.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:38 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3138560684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:38.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.401+0000 7fa72c6c3640 1 -- 172.21.15.112:0/1394131423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724101880 msgr2=0x7fa724101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.401+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/1394131423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724101880 0x7fa724101d00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa714009900 tx=0x7fa71402a410 comp rx=0 tx=0).stop 2024-04-24T23:17:38.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.401+0000 7fa72c6c3640 1 -- 172.21.15.112:0/1394131423 shutdown_connections 2024-04-24T23:17:38.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.401+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/1394131423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa724102240 0x7fa724108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.401+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/1394131423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724101880 0x7fa724101d00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.401+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/1394131423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa724100690 0x7fa724100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 -- 172.21.15.112:0/1394131423 >> 172.21.15.112:0/1394131423 conn(0x7fa7240fbc00 msgr2=0x7fa7240fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 -- 172.21.15.112:0/1394131423 shutdown_connections 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 -- 172.21.15.112:0/1394131423 wait complete. 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 Processor -- start 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 -- start start 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa724100690 0x7fa724197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa724101880 0x7fa7241980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724102240 0x7fa72419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa72410d550 con 0x7fa724102240 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa72410d3d0 con 0x7fa724101880 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72c6c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa72410d6d0 con 0x7fa724100690 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72ac60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724102240 0x7fa72419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72ac60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724102240 0x7fa72419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54754/0 (socket says 172.21.15.112:54754) 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.402+0000 7fa72ac60640 1 -- 172.21.15.112:0/601528535 learned_addr learned my addr 172.21.15.112:0/601528535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72a45f640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa724100690 0x7fa724197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa729c5e640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa724101880 0x7fa7241980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72ac60640 1 -- 172.21.15.112:0/601528535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa724100690 msgr2=0x7fa724197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72ac60640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa724100690 0x7fa724197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72ac60640 1 -- 172.21.15.112:0/601528535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa724101880 msgr2=0x7fa7241980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72ac60640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa724101880 0x7fa7241980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72ac60640 1 -- 172.21.15.112:0/601528535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72419f7c0 con 0x7fa724102240 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72ac60640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724102240 0x7fa72419f150 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00eaa0 tx=0x7fa71c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa71b7fe640 1 -- 172.21.15.112:0/601528535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00cdb0 con 0x7fa724102240 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa71b7fe640 1 -- 172.21.15.112:0/601528535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa71c004b20 con 0x7fa724102240 2024-04-24T23:17:38.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72c6c3640 1 -- 172.21.15.112:0/601528535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72419fab0 con 0x7fa724102240 2024-04-24T23:17:38.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa71b7fe640 1 -- 172.21.15.112:0/601528535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c011760 con 0x7fa724102240 2024-04-24T23:17:38.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.403+0000 7fa72c6c3640 1 -- 172.21.15.112:0/601528535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72419ffd0 con 0x7fa724102240 2024-04-24T23:17:38.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.405+0000 7fa71b7fe640 1 -- 172.21.15.112:0/601528535 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa71c004cd0 con 0x7fa724102240 2024-04-24T23:17:38.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.405+0000 7fa72f3cc740 1 Processor -- start 2024-04-24T23:17:38.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72f3cc740 1 -- start start 2024-04-24T23:17:38.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba0640 0x55612bba0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 0x55612bba5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 0x55612bba8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55612bb00670 con 0x55612bba0640 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55612baa8970 con 0x55612bba1030 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55612bafc630 con 0x55612bba61a0 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa71b7fe640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa7100a2b90 0x7fa7100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba0640 0x55612bba0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba0640 0x55612bba0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54762/0 (socket says 172.21.15.112:54762) 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 -- 172.21.15.112:0/1926919663 learned_addr learned my addr 172.21.15.112:0/1926919663 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72ac60640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa7100a2b90 0x7fa7100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 0x55612bba8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72a45f640 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 0x55612bba5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa71b7fe640 1 -- 172.21.15.112:0/601528535 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa71c0cf760 con 0x7fa724102240 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 -- 172.21.15.112:0/1926919663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 msgr2=0x55612bba8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 0x55612bba8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 -- 172.21.15.112:0/1926919663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 msgr2=0x55612bba5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 0x55612bba5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa729c5e640 1 -- 172.21.15.112:0/1926919663 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612bacc400 con 0x55612bba0640 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.406+0000 7fa72ac60640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa7100a2b90 0x7fa7100a5050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa71c01c000 tx=0x7fa71c06d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa729c5e640 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba0640 0x55612bba0a60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa7140314d0 tx=0x7fa7140058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf8be0169247ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa718ff9640 1 -- 172.21.15.112:0/1926919663 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714035070 con 0x55612bba0640 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa718ff9640 1 -- 172.21.15.112:0/1926919663 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa7140319d0 con 0x55612bba0640 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa718ff9640 1 -- 172.21.15.112:0/1926919663 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714004920 con 0x55612bba0640 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa72f3cc740 1 -- 172.21.15.112:0/1926919663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba0640 msgr2=0x55612bba0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba0640 0x55612bba0a60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa7140314d0 tx=0x7fa7140058d0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa72f3cc740 1 -- 172.21.15.112:0/1926919663 shutdown_connections 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 0x55612bba8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 0x55612bba5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/1926919663 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba0640 0x55612bba0a60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.407+0000 7fa72f3cc740 1 -- 172.21.15.112:0/1926919663 >> 172.21.15.112:0/1926919663 conn(0x55612bb978a0 msgr2=0x55612bb97ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 -- 172.21.15.112:0/1926919663 shutdown_connections 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 -- 172.21.15.112:0/1926919663 wait complete. 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 -- start start 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 Processor -- start 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 0x55612bbaac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba61a0 0x55612bbb1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55612bba90f0 con 0x55612bba61a0 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55612babc050 con 0x55612bba1030 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55612bba9270 con 0x55612bba0640 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72f3cc740 10 client.0 _ll_register_callbacks cb 0x55612b813be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba61a0 0x55612bbb1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba61a0 0x55612bbb1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54766/0 (socket says 172.21.15.112:54766) 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 -- 172.21.15.112:0/3255204229 learned_addr learned my addr 172.21.15.112:0/3255204229 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa72a45f640 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 0x55612bbaac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 -- 172.21.15.112:0/3255204229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 msgr2=0x55612bbaa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 -- 172.21.15.112:0/3255204229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 msgr2=0x55612bbaac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 0x55612bbaac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.408+0000 7fa729c5e640 1 -- 172.21.15.112:0/3255204229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612ba8ddc0 con 0x55612bba61a0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa729c5e640 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba61a0 0x55612bbb1cb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa7140096c0 tx=0x7fa714032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11af7e656a68cda2 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa70dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa70dffb640 1 -- 172.21.15.112:0/3255204229 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714035070 con 0x55612bba61a0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa70dffb640 1 -- 172.21.15.112:0/3255204229 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa7140046c0 con 0x55612bba61a0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3255204229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7140319d0 con 0x55612bba61a0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa70dffb640 1 -- 172.21.15.112:0/3255204229 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa714059050 con 0x55612bba61a0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 10 client.16374 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3255204229 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55612b8143f0 con 0x55612bba61a0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa70dffb640 1 -- 172.21.15.112:0/3255204229 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa7140319d0 con 0x55612bba61a0 2024-04-24T23:17:38.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa70dffb640 1 -- 172.21.15.112:0/3255204229 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7fa71403a040 con 0x55612bba61a0 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 10 client.16374 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 10 client.16374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 20 client.16374 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55612bbb61a0 0x55612bbb8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 4 client.16374 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.409+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3255204229 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55612bb0dbb0 con 0x55612bbb61a0 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.410+0000 7fa72a45f640 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55612bbb61a0 0x55612bbb8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.410+0000 7fa72a45f640 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55612bbb61a0 0x55612bbb8590 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5da10345f5392adf server_cookie=80e09c39b0e0e4dc in_seq=0 out_seq=0 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.410+0000 7fa70dffb640 10 client.16374 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa70dffb640 1 -- 172.21.15.112:0/3255204229 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55612bb0dbb0 con 0x55612bbb61a0 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa70dffb640 10 client.16374 handle_command_reply: tid=0 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 client.16374 shutdown 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3255204229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba61a0 msgr2=0x55612bbb1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba61a0 0x55612bbb1cb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa7140096c0 tx=0x7fa714032db0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3255204229 shutdown_connections 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55612bbb61a0 0x55612bbb8590 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba61a0 0x55612bbb1cb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa70dffb640 0 client.16374 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba1030 0x55612bbaac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3255204229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3255204229 >> 172.21.15.112:0/3255204229 conn(0x55612bb978a0 msgr2=0x55612bb96b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3255204229 shutdown_connections 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3255204229 wait complete. 2024-04-24T23:17:38.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.411+0000 7fa72f3cc740 20 client.16374 trim_cache size 0 max 16384 2024-04-24T23:17:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72f3cc740 1 Processor -- start 2024-04-24T23:17:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72f3cc740 1 -- start start 2024-04-24T23:17:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba0640 0x55612bb9b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbb1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 0x55612bbbcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241a84a0 con 0x55612bba1030 2024-04-24T23:17:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241aa1c0 con 0x55612bba0640 2024-04-24T23:17:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241a8b70 con 0x55612bba61a0 2024-04-24T23:17:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72a45f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbb1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72a45f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbb1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54778/0 (socket says 172.21.15.112:54778) 2024-04-24T23:17:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72a45f640 1 -- 172.21.15.112:0/4289306021 learned_addr learned my addr 172.21.15.112:0/4289306021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.424+0000 7fa72a45f640 1 -- 172.21.15.112:0/4289306021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 msgr2=0x55612bbbcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72a45f640 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 0x55612bbbcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72a45f640 1 -- 172.21.15.112:0/4289306021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba0640 msgr2=0x55612bb9b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa729c5e640 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba0640 0x55612bb9b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72a45f640 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba0640 0x55612bb9b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72a45f640 1 -- 172.21.15.112:0/4289306021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612b8de7d0 con 0x55612bba1030 2024-04-24T23:17:38.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa729c5e640 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba0640 0x55612bb9b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72a45f640 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbb1bd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55612bba1430 tx=0x7fa720012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac61c89fde03130 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa70ffff640 1 -- 172.21.15.112:0/4289306021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720021020 con 0x55612bba1030 2024-04-24T23:17:38.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa70ffff640 1 -- 172.21.15.112:0/4289306021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa72001d8b0 con 0x55612bba1030 2024-04-24T23:17:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa70ffff640 1 -- 172.21.15.112:0/4289306021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72000f420 con 0x55612bba1030 2024-04-24T23:17:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 -- 172.21.15.112:0/4289306021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 msgr2=0x55612bbb1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbb1bd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55612bba1430 tx=0x7fa720012820 comp rx=0 tx=0).stop 2024-04-24T23:17:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 -- 172.21.15.112:0/4289306021 shutdown_connections 2024-04-24T23:17:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba61a0 0x55612bbbcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbb1bd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/4289306021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba0640 0x55612bb9b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 -- 172.21.15.112:0/4289306021 >> 172.21.15.112:0/4289306021 conn(0x55612bb978a0 msgr2=0x55612bba25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 -- 172.21.15.112:0/4289306021 shutdown_connections 2024-04-24T23:17:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.425+0000 7fa72f3cc740 1 -- 172.21.15.112:0/4289306021 wait complete. 2024-04-24T23:17:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 1 -- start start 2024-04-24T23:17:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 1 Processor -- start 2024-04-24T23:17:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbafe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba61a0 0x55612bbb0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241a9a30 con 0x55612bba1030 2024-04-24T23:17:38.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241a92d0 con 0x55612bba61a0 2024-04-24T23:17:38.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7241a6450 con 0x55612bba0640 2024-04-24T23:17:38.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72f3cc740 10 client.0 _ll_register_callbacks cb 0x7fa720009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbafe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbafe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54784/0 (socket says 172.21.15.112:54784) 2024-04-24T23:17:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 -- 172.21.15.112:0/3557781537 learned_addr learned my addr 172.21.15.112:0/3557781537 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa729c5e640 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 -- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 msgr2=0x55612bbaf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 -- 172.21.15.112:0/3557781537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba61a0 msgr2=0x55612bbb0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba61a0 0x55612bbb0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 -- 172.21.15.112:0/3557781537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612bbb1430 con 0x55612bba1030 2024-04-24T23:17:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa729c5e640 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa72a45f640 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbafe30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa720005780 tx=0x7fa720011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabcdcb0ff891a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.426+0000 7fa70dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa70dffb640 1 -- 172.21.15.112:0/3557781537 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa720021030 con 0x55612bba1030 2024-04-24T23:17:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa70dffb640 1 -- 172.21.15.112:0/3557781537 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa720004240 con 0x55612bba1030 2024-04-24T23:17:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3557781537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa72001d8b0 con 0x55612bba1030 2024-04-24T23:17:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa70dffb640 1 -- 172.21.15.112:0/3557781537 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa720004440 con 0x55612bba1030 2024-04-24T23:17:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 10 client.16380 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3557781537 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55612bc1cab0 con 0x55612bba1030 2024-04-24T23:17:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa70dffb640 1 -- 172.21.15.112:0/3557781537 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa72001d8b0 con 0x55612bba1030 2024-04-24T23:17:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa70dffb640 1 -- 172.21.15.112:0/3557781537 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7fa720003bd0 con 0x55612bba1030 2024-04-24T23:17:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 10 client.16380 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 10 client.16380 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-24T23:17:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 20 client.16380 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55612bc1d4b0 0x55612bc1f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 4 client.16380 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3557781537 --> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55612bb07f00 con 0x55612bc1d4b0 2024-04-24T23:17:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.427+0000 7fa729c5e640 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55612bc1d4b0 0x55612bc1f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa70dffb640 10 client.16380 ms_handle_connect on v2:172.21.15.187:6838/3343164580 2024-04-24T23:17:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa729c5e640 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55612bc1d4b0 0x55612bc1f8d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=816b3377445515b1 server_cookie=d859d0047f6bdd36 in_seq=0 out_seq=0 2024-04-24T23:17:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa70dffb640 1 -- 172.21.15.112:0/3557781537 <== mds.1 v2:172.21.15.187:6838/3343164580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55612bb07f00 con 0x55612bc1d4b0 2024-04-24T23:17:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa70dffb640 10 client.16380 handle_command_reply: tid=0 2024-04-24T23:17:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 client.16380 shutdown 2024-04-24T23:17:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3557781537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 msgr2=0x55612bbafe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbafe30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa720005780 tx=0x7fa720011f20 comp rx=0 tx=0).stop 2024-04-24T23:17:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3557781537 shutdown_connections 2024-04-24T23:17:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:6838/3343164580,v1:172.21.15.187:6839/3343164580] conn(0x55612bc1d4b0 0x55612bc1f8d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55612bba61a0 0x55612bbb0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa70dffb640 0 client.16380 ms_handle_reset on v2:172.21.15.187:6838/3343164580 2024-04-24T23:17:38.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55612bba1030 0x55612bbafe30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 --2- 172.21.15.112:0/3557781537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55612bba0640 0x55612bbaf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3557781537 >> 172.21.15.112:0/3557781537 conn(0x55612bb978a0 msgr2=0x55612bb9b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3557781537 shutdown_connections 2024-04-24T23:17:38.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 1 -- 172.21.15.112:0/3557781537 wait complete. 2024-04-24T23:17:38.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.428+0000 7fa72f3cc740 20 client.16380 trim_cache size 0 max 16384 2024-04-24T23:17:38.450 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:38.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 -- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa7100a2b90 msgr2=0x7fa7100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa7100a2b90 0x7fa7100a5050 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa71c01c000 tx=0x7fa71c06d040 comp rx=0 tx=0).stop 2024-04-24T23:17:38.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 -- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724102240 msgr2=0x7fa72419f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724102240 0x7fa72419f150 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00eaa0 tx=0x7fa71c00ef60 comp rx=0 tx=0).stop 2024-04-24T23:17:38.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 -- 172.21.15.112:0/601528535 shutdown_connections 2024-04-24T23:17:38.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa7100a2b90 0x7fa7100a5050 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa724102240 0x7fa72419f150 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa724101880 0x7fa7241980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 --2- 172.21.15.112:0/601528535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa724100690 0x7fa724197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 -- 172.21.15.112:0/601528535 >> 172.21.15.112:0/601528535 conn(0x7fa7240fbc00 msgr2=0x7fa724104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 -- 172.21.15.112:0/601528535 shutdown_connections 2024-04-24T23:17:38.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.429+0000 7fa72c6c3640 1 -- 172.21.15.112:0/601528535 wait complete. 2024-04-24T23:17:38.455 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-24T23:17:38.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:38 smithi187 ceph-mon[33292]: pgmap v449: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 475 KiB/s wr, 144 op/s 2024-04-24T23:17:38.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:38 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3138560684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:17:38.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 -- 172.21.15.112:0/3620919716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4101870 msgr2=0x7f3bc4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 --2- 172.21.15.112:0/3620919716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4101870 0x7f3bc4101cf0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009900 tx=0x7f3bb802a410 comp rx=0 tx=0).stop 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 -- 172.21.15.112:0/3620919716 shutdown_connections 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 --2- 172.21.15.112:0/3620919716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bc4102230 0x7f3bc4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 --2- 172.21.15.112:0/3620919716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4101870 0x7f3bc4101cf0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 --2- 172.21.15.112:0/3620919716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc4100680 0x7f3bc4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 -- 172.21.15.112:0/3620919716 >> 172.21.15.112:0/3620919716 conn(0x7f3bc40fbbf0 msgr2=0x7f3bc40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 -- 172.21.15.112:0/3620919716 shutdown_connections 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.553+0000 7f3bca2df640 1 -- 172.21.15.112:0/3620919716 wait complete. 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bca2df640 1 Processor -- start 2024-04-24T23:17:38.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bca2df640 1 -- start start 2024-04-24T23:17:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bca2df640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc4100680 0x7f3bc4196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bca2df640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bc4101870 0x7f3bc4196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bca2df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102230 0x7f3bc419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bca2df640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410c430 con 0x7f3bc4102230 2024-04-24T23:17:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bca2df640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410c2b0 con 0x7f3bc4101870 2024-04-24T23:17:38.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bca2df640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410c5b0 con 0x7f3bc4100680 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bc887c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102230 0x7f3bc419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bc887c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102230 0x7f3bc419e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54804/0 (socket says 172.21.15.112:54804) 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bc887c640 1 -- 172.21.15.112:0/2971499421 learned_addr learned my addr 172.21.15.112:0/2971499421 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.554+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bc4101870 0x7f3bc4196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc4100680 0x7f3bc4196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc887c640 1 -- 172.21.15.112:0/2971499421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc4100680 msgr2=0x7f3bc4196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc887c640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc4100680 0x7f3bc4196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc887c640 1 -- 172.21.15.112:0/2971499421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bc4101870 msgr2=0x7f3bc4196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc887c640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bc4101870 0x7f3bc4196f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc887c640 1 -- 172.21.15.112:0/2971499421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc419e670 con 0x7f3bc4102230 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc4100680 0x7f3bc4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc887c640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102230 0x7f3bc419e000 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00cad0 tx=0x7f3bbc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc17fa640 1 -- 172.21.15.112:0/2971499421 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00ae20 con 0x7f3bc4102230 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bca2df640 1 -- 172.21.15.112:0/2971499421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc419e960 con 0x7f3bc4102230 2024-04-24T23:17:38.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc17fa640 1 -- 172.21.15.112:0/2971499421 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3bbc004b20 con 0x7f3bc4102230 2024-04-24T23:17:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bc17fa640 1 -- 172.21.15.112:0/2971499421 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00f780 con 0x7f3bc4102230 2024-04-24T23:17:38.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.555+0000 7f3bca2df640 1 -- 172.21.15.112:0/2971499421 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc419ee80 con 0x7f3bc4102230 2024-04-24T23:17:38.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.557+0000 7f3bc17fa640 1 -- 172.21.15.112:0/2971499421 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3bbc004cd0 con 0x7f3bc4102230 2024-04-24T23:17:38.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bccfdf740 1 Processor -- start 2024-04-24T23:17:38.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bccfdf740 1 -- start start 2024-04-24T23:17:38.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656aa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd6560e670 con 0x55fd656aa230 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fd655b6970 con 0x55fd656afd90 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fd6560a630 con 0x55fd656aac20 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc17fa640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3bac0a2b90 0x7f3bac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656aa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54812/0 (socket says 172.21.15.112:54812) 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 -- 172.21.15.112:0/1774245027 learned_addr learned my addr 172.21.15.112:0/1774245027 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/1774245027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc887c640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3bac0a2b90 0x7f3bac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 -- 172.21.15.112:0/1774245027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 msgr2=0x55fd656af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/1774245027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 -- 172.21.15.112:0/1774245027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 msgr2=0x55fd656b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/1774245027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 -- 172.21.15.112:0/1774245027 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd655da400 con 0x55fd656aa230 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc17fa640 1 -- 172.21.15.112:0/2971499421 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f3bbc0cf4d0 con 0x7f3bc4102230 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.558+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/1774245027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656aa650 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009900 tx=0x7f3bb80318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbbb75b9d18dcc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3baa7fc640 1 -- 172.21.15.112:0/1774245027 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8040070 con 0x55fd656aa230 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3baa7fc640 1 -- 172.21.15.112:0/1774245027 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3bb8005520 con 0x55fd656aa230 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bc887c640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3bac0a2b90 0x7f3bac0a5050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc097da0 tx=0x7f3bbc06a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3baa7fc640 1 -- 172.21.15.112:0/1774245027 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8035040 con 0x55fd656aa230 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 -- 172.21.15.112:0/1774245027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 msgr2=0x55fd656aa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/1774245027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656aa650 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009900 tx=0x7f3bb80318c0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 -- 172.21.15.112:0/1774245027 shutdown_connections 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/1774245027 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/1774245027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/1774245027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656aa650 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 -- 172.21.15.112:0/1774245027 >> 172.21.15.112:0/1774245027 conn(0x55fd656a58a0 msgr2=0x55fd656a5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 -- 172.21.15.112:0/1774245027 shutdown_connections 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.559+0000 7f3bccfdf740 1 -- 172.21.15.112:0/1774245027 wait complete. 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 1 -- start start 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 1 Processor -- start 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656aa230 0x55fd656b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656afd90 0x55fd656bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fd656b2ce0 con 0x55fd656afd90 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x55fd655ca050 con 0x55fd656aa230 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fd656b2e60 con 0x55fd656aac20 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bccfdf740 10 client.0 _ll_register_callbacks cb 0x55fd65321be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656afd90 0x55fd656bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656afd90 0x55fd656bb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54828/0 (socket says 172.21.15.112:54828) 2024-04-24T23:17:38.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 -- 172.21.15.112:0/2730710885 learned_addr learned my addr 172.21.15.112:0/2730710885 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656aa230 0x55fd656b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 -- 172.21.15.112:0/2730710885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 msgr2=0x55fd656b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 -- 172.21.15.112:0/2730710885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656aa230 msgr2=0x55fd656b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656aa230 0x55fd656b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc37fe640 1 -- 172.21.15.112:0/2730710885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd6559bdc0 con 0x55fd656afd90 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.560+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656afd90 0x55fd656bb8b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000c00 tx=0x7f3bb80047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e81302a0d3959f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3ba37fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2730710885 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8040070 con 0x55fd656afd90 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2730710885 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f3bb80051e0 con 0x55fd656afd90 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2730710885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb8005520 con 0x55fd656afd90 2024-04-24T23:17:38.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2730710885 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3bb8035070 con 0x55fd656afd90 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 10 client.16392 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2730710885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd653223f0 con 0x55fd656afd90 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2730710885 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3bb8005520 con 0x55fd656afd90 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2730710885 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7f3bb8003cc0 con 0x55fd656afd90 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 10 client.16392 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 20 client.16392 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55fd656bfda0 0x55fd656c2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 4 client.16392 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2730710885 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd6561bbb0 con 0x55fd656bfda0 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.561+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55fd656bfda0 0x55fd656c2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.562+0000 7f3ba37fe640 10 client.16392 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.562+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55fd656bfda0 0x55fd656c2190 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7075046465dedf server_cookie=7897aaa0782937d6 in_seq=0 out_seq=0 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2730710885 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd6561bbb0 con 0x55fd656bfda0 2024-04-24T23:17:38.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3ba37fe640 10 client.16392 handle_command_reply: tid=0 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 client.16392 shutdown 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2730710885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656afd90 msgr2=0x55fd656bb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656afd90 0x55fd656bb8b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8000c00 tx=0x7f3bb80047a0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2730710885 shutdown_connections 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55fd656bfda0 0x55fd656c2190 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656afd90 0x55fd656bb8b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3ba37fe640 0 client.16392 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2730710885 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656aa230 0x55fd656b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2730710885 >> 172.21.15.112:0/2730710885 conn(0x55fd656a58a0 msgr2=0x55fd656a4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2730710885 shutdown_connections 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2730710885 wait complete. 2024-04-24T23:17:38.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.563+0000 7f3bccfdf740 20 client.16392 trim_cache size 0 max 16384 2024-04-24T23:17:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bccfdf740 1 Processor -- start 2024-04-24T23:17:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bccfdf740 1 -- start start 2024-04-24T23:17:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41aa5c0 con 0x55fd656aa230 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41ac2e0 con 0x55fd656afd90 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41aac90 con 0x55fd656aac20 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54840/0 (socket says 172.21.15.112:54840) 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 -- 172.21.15.112:0/3276751038 learned_addr learned my addr 172.21.15.112:0/3276751038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 -- 172.21.15.112:0/3276751038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 msgr2=0x55fd656ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 -- 172.21.15.112:0/3276751038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 msgr2=0x55fd656bb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 -- 172.21.15.112:0/3276751038 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd653ec7d0 con 0x55fd656aa230 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.576+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656abdd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8031570 tx=0x7f3bb8004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc3ab4d9b398344 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3ba9ffb640 1 -- 172.21.15.112:0/3276751038 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8040040 con 0x55fd656aa230 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3ba9ffb640 1 -- 172.21.15.112:0/3276751038 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3bb803c6d0 con 0x55fd656aa230 2024-04-24T23:17:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3ba9ffb640 1 -- 172.21.15.112:0/3276751038 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8004a70 con 0x55fd656aa230 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 -- 172.21.15.112:0/3276751038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 msgr2=0x55fd656abdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656abdd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8031570 tx=0x7f3bb8004f70 comp rx=0 tx=0).stop 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 -- 172.21.15.112:0/3276751038 shutdown_connections 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656bb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aac20 0x55fd656ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/3276751038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aa230 0x55fd656abdd0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 -- 172.21.15.112:0/3276751038 >> 172.21.15.112:0/3276751038 conn(0x55fd656a58a0 msgr2=0x55fd656ab6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 -- 172.21.15.112:0/3276751038 shutdown_connections 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 -- 172.21.15.112:0/3276751038 wait complete. 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.577+0000 7f3bccfdf740 1 -- start start 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bccfdf740 1 Processor -- start 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aa230 0x55fd656b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aac20 0x55fd656b9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bccfdf740 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656ba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41abb50 con 0x55fd656aac20 2024-04-24T23:17:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41ab3f0 con 0x55fd656afd90 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bccfdf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41a8570 con 0x55fd656aa230 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bccfdf740 10 client.0 _ll_register_callbacks cb 0x7f3bb80317f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aac20 0x55fd656b9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aac20 0x55fd656b9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54844/0 (socket says 172.21.15.112:54844) 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 -- 172.21.15.112:0/2764386193 learned_addr learned my addr 172.21.15.112:0/2764386193 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aa230 0x55fd656b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 -- 172.21.15.112:0/2764386193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aa230 msgr2=0x55fd656b92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aa230 0x55fd656b92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 -- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 msgr2=0x55fd656ba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656ba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 -- 172.21.15.112:0/2764386193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd656bae20 con 0x55fd656aac20 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:17:38.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3bc3fff640 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aac20 0x55fd656b9820 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x55fd656ab020 tx=0x7f3bb4011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5c0834aacf9831 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.578+0000 7f3ba37fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2764386193 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb401e070 con 0x55fd656aac20 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2764386193 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f3bb401a800 con 0x55fd656aac20 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2764386193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bb803c6d0 con 0x55fd656aac20 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2764386193 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3bb402a050 con 0x55fd656aac20 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 10 client.16398 fetch_fsmap learned FSMap version 38 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2764386193 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd65726560 con 0x55fd656aac20 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2764386193 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f3bb803c6d0 con 0x55fd656aac20 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2764386193 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8418+0+0 (secure 0 0 0) 0x7f3bb40227b0 con 0x55fd656aac20 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 10 client.16398 fetch_fsmap finished waiting for FSMap version 38 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 20 client.16398 populate_metadata read hostname 'smithi112' 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55fd65727050 0x55fd65729470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 4 client.16398 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2764386193 --> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd65615f00 con 0x55fd65727050 2024-04-24T23:17:38.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.579+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55fd65727050 0x55fd65729470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bc37fe640 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55fd65727050 0x55fd65729470 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7a380a5201b5e6f server_cookie=14d8f3d38b7c0cac in_seq=0 out_seq=0 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3ba37fe640 10 client.16398 ms_handle_connect on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3ba37fe640 1 -- 172.21.15.112:0/2764386193 <== mds.0 v2:172.21.15.150:6838/748900426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd65615f00 con 0x55fd65727050 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3ba37fe640 10 client.16398 handle_command_reply: tid=0 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 client.16398 shutdown 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2764386193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aac20 msgr2=0x55fd656b9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aac20 0x55fd656b9820 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x55fd656ab020 tx=0x7f3bb4011ae0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2764386193 shutdown_connections 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:6838/748900426,v1:172.21.15.150:6839/748900426] conn(0x55fd65727050 0x55fd65729470 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55fd656afd90 0x55fd656ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fd656aac20 0x55fd656b9820 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 --2- 172.21.15.112:0/2764386193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd656aa230 0x55fd656b92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3ba37fe640 0 client.16398 ms_handle_reset on v2:172.21.15.150:6838/748900426 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2764386193 >> 172.21.15.112:0/2764386193 conn(0x55fd656a58a0 msgr2=0x55fd656abf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2764386193 shutdown_connections 2024-04-24T23:17:38.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 1 -- 172.21.15.112:0/2764386193 wait complete. 2024-04-24T23:17:38.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.580+0000 7f3bccfdf740 20 client.16398 trim_cache size 0 max 16384 2024-04-24T23:17:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:[] 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 -- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3bac0a2b90 msgr2=0x7f3bac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3bac0a2b90 0x7f3bac0a5050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc097da0 tx=0x7f3bbc06a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 -- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102230 msgr2=0x7f3bc419e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102230 0x7f3bc419e000 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00cad0 tx=0x7f3bbc00cf90 comp rx=0 tx=0).stop 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 -- 172.21.15.112:0/2971499421 shutdown_connections 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f3bac0a2b90 0x7f3bac0a5050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bc4102230 0x7f3bc419e000 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bc4101870 0x7f3bc4196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 --2- 172.21.15.112:0/2971499421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bc4100680 0x7f3bc4196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 -- 172.21.15.112:0/2971499421 >> 172.21.15.112:0/2971499421 conn(0x7f3bc40fbbf0 msgr2=0x7f3bc4104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 -- 172.21.15.112:0/2971499421 shutdown_connections 2024-04-24T23:17:38.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:38.581+0000 7f3bca2df640 1 -- 172.21.15.112:0/2971499421 wait complete. 2024-04-24T23:17:39.594 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-24T23:17:39.594 INFO:tasks.fwd_scrub:done joining 2024-04-24T23:17:39.595 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-24T23:17:39.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:17:39.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- 172.21.15.112:0/1814066946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 msgr2=0x7fa424100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:39.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/1814066946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 0x7fa424100aa0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa414009900 tx=0x7fa41402a410 comp rx=0 tx=0).stop 2024-04-24T23:17:39.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- 172.21.15.112:0/1814066946 shutdown_connections 2024-04-24T23:17:39.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/1814066946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa424102230 0x7fa424108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/1814066946 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa424101870 0x7fa424101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/1814066946 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 0x7fa424100aa0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- 172.21.15.112:0/1814066946 >> 172.21.15.112:0/1814066946 conn(0x7fa4240fbbf0 msgr2=0x7fa4240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:39.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- 172.21.15.112:0/1814066946 shutdown_connections 2024-04-24T23:17:39.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- 172.21.15.112:0/1814066946 wait complete. 2024-04-24T23:17:39.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 Processor -- start 2024-04-24T23:17:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- start start 2024-04-24T23:17:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 0x7fa4240694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa424101870 0x7fa42419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa424102230 0x7fa42419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:39.726 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa42410c480 con 0x7fa424100680 2024-04-24T23:17:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa42410c300 con 0x7fa424101870 2024-04-24T23:17:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.713+0000 7fa42c5d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa42410c600 con 0x7fa424102230 2024-04-24T23:17:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 0x7fa4240694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 0x7fa4240694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54870/0 (socket says 172.21.15.112:54870) 2024-04-24T23:17:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 -- 172.21.15.112:0/4088876295 learned_addr learned my addr 172.21.15.112:0/4088876295 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:39.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa429b70640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa424101870 0x7fa42419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42ab72640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa424102230 0x7fa42419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 -- 172.21.15.112:0/4088876295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa424102230 msgr2=0x7fa42419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:39.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa424102230 0x7fa42419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 -- 172.21.15.112:0/4088876295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa424101870 msgr2=0x7fa42419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa424101870 0x7fa42419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 -- 172.21.15.112:0/4088876295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42419f780 con 0x7fa424100680 2024-04-24T23:17:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42ab72640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa424102230 0x7fa42419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:39.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42a371640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 0x7fa4240694a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa4140316a0 tx=0x7fa414032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa41b7fe640 1 -- 172.21.15.112:0/4088876295 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41403f030 con 0x7fa424100680 2024-04-24T23:17:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa41b7fe640 1 -- 172.21.15.112:0/4088876295 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa414003d00 con 0x7fa424100680 2024-04-24T23:17:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa41b7fe640 1 -- 172.21.15.112:0/4088876295 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414035070 con 0x7fa424100680 2024-04-24T23:17:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42419fa10 con 0x7fa424100680 2024-04-24T23:17:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.714+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42419ff00 con 0x7fa424100680 2024-04-24T23:17:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.715+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4241079a0 con 0x7fa424100680 2024-04-24T23:17:39.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.717+0000 7fa41b7fe640 1 -- 172.21.15.112:0/4088876295 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa414042030 con 0x7fa424100680 2024-04-24T23:17:39.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.717+0000 7fa41b7fe640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa4100a2b90 0x7fa4100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:39.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.717+0000 7fa429b70640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa4100a2b90 0x7fa4100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:39.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.717+0000 7fa41b7fe640 1 -- 172.21.15.112:0/4088876295 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa4140ef9a0 con 0x7fa424100680 2024-04-24T23:17:39.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.718+0000 7fa429b70640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa4100a2b90 0x7fa4100a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa424069210 tx=0x7fa42000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:39.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.722+0000 7fa41b7fe640 1 -- 172.21.15.112:0/4088876295 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa4140bc510 con 0x7fa424100680 2024-04-24T23:17:39.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.900+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa424062340 con 0x7fa424100680 2024-04-24T23:17:39.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.902+0000 7fa41b7fe640 1 -- 172.21.15.112:0/4088876295 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7fa4140bbe10 con 0x7fa424100680 2024-04-24T23:17:39.904 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:17:39.904 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:17:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa4100a2b90 msgr2=0x7fa4100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa4100a2b90 0x7fa4100a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa424069210 tx=0x7fa42000a2d0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 msgr2=0x7fa4240694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 0x7fa4240694a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa4140316a0 tx=0x7fa414032d70 comp rx=0 tx=0).stop 2024-04-24T23:17:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 shutdown_connections 2024-04-24T23:17:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa4100a2b90 0x7fa4100a5050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa424102230 0x7fa42419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa424101870 0x7fa42419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 --2- 172.21.15.112:0/4088876295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa424100680 0x7fa4240694a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 >> 172.21.15.112:0/4088876295 conn(0x7fa4240fbbf0 msgr2=0x7fa424104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 shutdown_connections 2024-04-24T23:17:39.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:39.904+0000 7fa42c5d5640 1 -- 172.21.15.112:0/4088876295 wait complete. 2024-04-24T23:17:39.917 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:17:39.918 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 -- 172.21.15.112:0/235461481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde641006a0 msgr2=0x7fde64100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 --2- 172.21.15.112:0/235461481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde641006a0 0x7fde64100ac0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fde54009900 tx=0x7fde5402a410 comp rx=0 tx=0).stop 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 -- 172.21.15.112:0/235461481 shutdown_connections 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 --2- 172.21.15.112:0/235461481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde64102250 0x7fde64108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 --2- 172.21.15.112:0/235461481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64101890 0x7fde64101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 --2- 172.21.15.112:0/235461481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde641006a0 0x7fde64100ac0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 -- 172.21.15.112:0/235461481 >> 172.21.15.112:0/235461481 conn(0x7fde640fbc30 msgr2=0x7fde640fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 -- 172.21.15.112:0/235461481 shutdown_connections 2024-04-24T23:17:40.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 -- 172.21.15.112:0/235461481 wait complete. 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 Processor -- start 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.019+0000 7fde69a1c640 1 -- start start 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde69a1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde641006a0 0x7fde64196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde69a1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde64101890 0x7fde64197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde69a1c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64102250 0x7fde6419e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde69a1c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde6410c450 con 0x7fde64101890 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde69a1c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fde6410c2d0 con 0x7fde64102250 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde69a1c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde6410c5d0 con 0x7fde641006a0 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde64101890 0x7fde64197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde64101890 0x7fde64197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54896/0 (socket says 172.21.15.112:54896) 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 -- 172.21.15.112:0/1512502912 learned_addr learned my addr 172.21.15.112:0/1512502912 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde637fe640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64102250 0x7fde6419e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde62ffd640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde641006a0 0x7fde64196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 -- 172.21.15.112:0/1512502912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde641006a0 msgr2=0x7fde64196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde641006a0 0x7fde64196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 -- 172.21.15.112:0/1512502912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64102250 msgr2=0x7fde6419e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64102250 0x7fde6419e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 -- 172.21.15.112:0/1512502912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6419e740 con 0x7fde64101890 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde5a7fc640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde64101890 0x7fde64197060 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00e9f0 tx=0x7fde4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde68a1a640 1 -- 172.21.15.112:0/1512502912 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c00cd80 con 0x7fde64101890 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6419ea30 con 0x7fde64101890 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde68a1a640 1 -- 172.21.15.112:0/1512502912 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fde4c010930 con 0x7fde64101890 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde68a1a640 1 -- 172.21.15.112:0/1512502912 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c018b00 con 0x7fde64101890 2024-04-24T23:17:40.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.020+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6419ef80 con 0x7fde64101890 2024-04-24T23:17:40.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.022+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde64107a90 con 0x7fde64101890 2024-04-24T23:17:40.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.023+0000 7fde68a1a640 1 -- 172.21.15.112:0/1512502912 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fde4c018ca0 con 0x7fde64101890 2024-04-24T23:17:40.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.023+0000 7fde68a1a640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fde500a2b90 0x7fde500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.023+0000 7fde62ffd640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fde500a2b90 0x7fde500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.023+0000 7fde68a1a640 1 -- 172.21.15.112:0/1512502912 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fde4c0cfaa0 con 0x7fde64101890 2024-04-24T23:17:40.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.024+0000 7fde62ffd640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fde500a2b90 0x7fde500a5050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fde54009230 tx=0x7fde54031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.027+0000 7fde68a1a640 1 -- 172.21.15.112:0/1512502912 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fde4c09bfc0 con 0x7fde64101890 2024-04-24T23:17:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.205+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fde6402d3c0 con 0x7fde64101890 2024-04-24T23:17:40.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.206+0000 7fde68a1a640 1 -- 172.21.15.112:0/1512502912 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fde4c0a02c0 con 0x7fde64101890 2024-04-24T23:17:40.209 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:17:40.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fde500a2b90 msgr2=0x7fde500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fde500a2b90 0x7fde500a5050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fde54009230 tx=0x7fde54031640 comp rx=0 tx=0).stop 2024-04-24T23:17:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde64101890 msgr2=0x7fde64197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde64101890 0x7fde64197060 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00e9f0 tx=0x7fde4c00eeb0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 shutdown_connections 2024-04-24T23:17:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fde500a2b90 0x7fde500a5050 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde64102250 0x7fde6419e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde64101890 0x7fde64197060 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 --2- 172.21.15.112:0/1512502912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde641006a0 0x7fde64196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 >> 172.21.15.112:0/1512502912 conn(0x7fde640fbc30 msgr2=0x7fde64104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 shutdown_connections 2024-04-24T23:17:40.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.208+0000 7fde69a1c640 1 -- 172.21.15.112:0/1512502912 wait complete. 2024-04-24T23:17:40.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:40 smithi112 ceph-mon[29186]: pgmap v450: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-04-24T23:17:40.220 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:40 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/4088876295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:17:40.221 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:17:40.289 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:40 smithi187 ceph-mon[33292]: pgmap v450: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-04-24T23:17:40.289 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:40 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/4088876295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:17:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:40 smithi150 ceph-mon[32247]: pgmap v450: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 11 KiB/s wr, 37 op/s 2024-04-24T23:17:40.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:40 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/4088876295' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:17:40.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.320+0000 7f07ee49a640 1 -- 172.21.15.112:0/3689850735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e81022a0 msgr2=0x7f07e8104690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.320+0000 7f07ee49a640 1 --2- 172.21.15.112:0/3689850735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e81022a0 0x7f07e8104690 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009900 tx=0x7f07dc02a410 comp rx=0 tx=0).stop 2024-04-24T23:17:40.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.320+0000 7f07ee49a640 1 -- 172.21.15.112:0/3689850735 shutdown_connections 2024-04-24T23:17:40.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.320+0000 7f07ee49a640 1 --2- 172.21.15.112:0/3689850735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e8107500 0x7f07e81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.320+0000 7f07ee49a640 1 --2- 172.21.15.112:0/3689850735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e8104bd0 0x7f07e8106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.320+0000 7f07ee49a640 1 --2- 172.21.15.112:0/3689850735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e81022a0 0x7f07e8104690 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.320+0000 7f07ee49a640 1 -- 172.21.15.112:0/3689850735 >> 172.21.15.112:0/3689850735 conn(0x7f07e80fbbf0 msgr2=0x7f07e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 -- 172.21.15.112:0/3689850735 shutdown_connections 2024-04-24T23:17:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 -- 172.21.15.112:0/3689850735 wait complete. 2024-04-24T23:17:40.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 Processor -- start 2024-04-24T23:17:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 -- start start 2024-04-24T23:17:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e81022a0 0x7f07e8197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e8104bd0 0x7f07e81980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e8107500 0x7f07e819f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f07e810d550 con 0x7f07e8104bd0 2024-04-24T23:17:40.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f07e810d3d0 con 0x7f07e8107500 2024-04-24T23:17:40.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07ee49a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07e810d6d0 con 0x7f07e81022a0 2024-04-24T23:17:40.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e8104bd0 0x7f07e81980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e8104bd0 0x7f07e81980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54914/0 (socket says 172.21.15.112:54914) 2024-04-24T23:17:40.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.321+0000 7f07e77fe640 1 -- 172.21.15.112:0/2763970798 learned_addr learned my addr 172.21.15.112:0/2763970798 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:40.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e7fff640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e81022a0 0x7f07e8197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07eca37640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e8107500 0x7f07e819f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e77fe640 1 -- 172.21.15.112:0/2763970798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e81022a0 msgr2=0x7f07e8197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e77fe640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e81022a0 0x7f07e8197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e77fe640 1 -- 172.21.15.112:0/2763970798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e8107500 msgr2=0x7f07e819f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e77fe640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e8107500 0x7f07e819f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e77fe640 1 -- 172.21.15.112:0/2763970798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e819f790 con 0x7f07e8104bd0 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e7fff640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e81022a0 0x7f07e8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e77fe640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e8104bd0 0x7f07e81980b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f07d8005fd0 tx=0x7f07d800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e57fa640 1 -- 172.21.15.112:0/2763970798 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8017070 con 0x7f07e8104bd0 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e57fa640 1 -- 172.21.15.112:0/2763970798 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f07d8009c30 con 0x7f07e8104bd0 2024-04-24T23:17:40.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e819fa80 con 0x7f07e8104bd0 2024-04-24T23:17:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07e57fa640 1 -- 172.21.15.112:0/2763970798 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8010460 con 0x7f07e8104bd0 2024-04-24T23:17:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.322+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e819ffd0 con 0x7f07e8104bd0 2024-04-24T23:17:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.323+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e80fd6e0 con 0x7f07e8104bd0 2024-04-24T23:17:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.324+0000 7f07e57fa640 1 -- 172.21.15.112:0/2763970798 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f07d8003eb0 con 0x7f07e8104bd0 2024-04-24T23:17:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.324+0000 7f07e57fa640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07d00a2b90 0x7f07d00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.325+0000 7f07e7fff640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07d00a2b90 0x7f07d00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.325+0000 7f07e57fa640 1 -- 172.21.15.112:0/2763970798 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f07d80cf1f0 con 0x7f07e8104bd0 2024-04-24T23:17:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.325+0000 7f07e7fff640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07d00a2b90 0x7f07d00a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f07dc000c00 tx=0x7f07dc031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.329+0000 7f07e57fa640 1 -- 172.21.15.112:0/2763970798 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f07d8020030 con 0x7f07e8104bd0 2024-04-24T23:17:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.504+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f07e8062340 con 0x7f07e8104bd0 2024-04-24T23:17:40.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.505+0000 7f07e57fa640 1 -- 172.21.15.112:0/2763970798 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f07d809bd60 con 0x7f07e8104bd0 2024-04-24T23:17:40.507 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07d00a2b90 msgr2=0x7f07d00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07d00a2b90 0x7f07d00a5050 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f07dc000c00 tx=0x7f07dc031640 comp rx=0 tx=0).stop 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e8104bd0 msgr2=0x7f07e81980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e8104bd0 0x7f07e81980b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f07d8005fd0 tx=0x7f07d800dde0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 shutdown_connections 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f07d00a2b90 0x7f07d00a5050 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07e8107500 0x7f07e819f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07e8104bd0 0x7f07e81980b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 --2- 172.21.15.112:0/2763970798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07e81022a0 0x7f07e8197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 >> 172.21.15.112:0/2763970798 conn(0x7f07e80fbbf0 msgr2=0x7f07e8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 shutdown_connections 2024-04-24T23:17:40.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.507+0000 7f07ee49a640 1 -- 172.21.15.112:0/2763970798 wait complete. 2024-04-24T23:17:40.520 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 -- 172.21.15.112:0/773272065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4101870 msgr2=0x7fedc4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 --2- 172.21.15.112:0/773272065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4101870 0x7fedc4101cf0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009900 tx=0x7fedb802a410 comp rx=0 tx=0).stop 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 -- 172.21.15.112:0/773272065 shutdown_connections 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 --2- 172.21.15.112:0/773272065 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc4102230 0x7fedc4108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 --2- 172.21.15.112:0/773272065 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4101870 0x7fedc4101cf0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 --2- 172.21.15.112:0/773272065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc4100680 0x7fedc4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 -- 172.21.15.112:0/773272065 >> 172.21.15.112:0/773272065 conn(0x7fedc40fbbf0 msgr2=0x7fedc40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 -- 172.21.15.112:0/773272065 shutdown_connections 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.618+0000 7fedc907d640 1 -- 172.21.15.112:0/773272065 wait complete. 2024-04-24T23:17:40.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.619+0000 7fedc907d640 1 Processor -- start 2024-04-24T23:17:40.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.619+0000 7fedc907d640 1 -- start start 2024-04-24T23:17:40.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.619+0000 7fedc907d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc4100680 0x7fedc4199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.619+0000 7fedc907d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc4101870 0x7fedc419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.619+0000 7fedc907d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4102230 0x7fedc41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.619+0000 7fedc907d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fedc410d540 con 0x7fedc4102230 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.619+0000 7fedc907d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fedc410d3c0 con 0x7fedc4101870 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.619+0000 7fedc907d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fedc410d6c0 con 0x7fedc4100680 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4102230 0x7fedc41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4102230 0x7fedc41a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54046/0 (socket says 172.21.15.112:54046) 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 -- 172.21.15.112:0/2807905806 learned_addr learned my addr 172.21.15.112:0/2807905806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc259c640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc4101870 0x7fedc419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc2d9d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc4100680 0x7fedc4199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 -- 172.21.15.112:0/2807905806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc4100680 msgr2=0x7fedc4199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc4100680 0x7fedc4199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 -- 172.21.15.112:0/2807905806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc4101870 msgr2=0x7fedc419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc4101870 0x7fedc419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 -- 172.21.15.112:0/2807905806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc41a19c0 con 0x7fedc4102230 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc359e640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4102230 0x7fedc41a1320 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00e970 tx=0x7fedbc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedabfff640 1 -- 172.21.15.112:0/2807905806 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00ccc0 con 0x7fedc4102230 2024-04-24T23:17:40.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc41a1cb0 con 0x7fedc4102230 2024-04-24T23:17:40.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedabfff640 1 -- 172.21.15.112:0/2807905806 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fedbc00ce60 con 0x7fedc4102230 2024-04-24T23:17:40.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedabfff640 1 -- 172.21.15.112:0/2807905806 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc010690 con 0x7fedc4102230 2024-04-24T23:17:40.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.620+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc41a21b0 con 0x7fedc4102230 2024-04-24T23:17:40.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.621+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc41a23e0 con 0x7fedc4102230 2024-04-24T23:17:40.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.622+0000 7fedabfff640 1 -- 172.21.15.112:0/2807905806 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fedbc0046c0 con 0x7fedc4102230 2024-04-24T23:17:40.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.623+0000 7fedabfff640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fedac0a2b90 0x7fedac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.623+0000 7fedc2d9d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fedac0a2b90 0x7fedac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.623+0000 7fedabfff640 1 -- 172.21.15.112:0/2807905806 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fedbc0cf8e0 con 0x7fedc4102230 2024-04-24T23:17:40.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.623+0000 7fedc2d9d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fedac0a2b90 0x7fedac0a5050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fedc419b280 tx=0x7fedb4009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.627+0000 7fedabfff640 1 -- 172.21.15.112:0/2807905806 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fedbc09c450 con 0x7fedc4102230 2024-04-24T23:17:40.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.804+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fedc41a25b0 con 0x7fedc4102230 2024-04-24T23:17:40.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.805+0000 7fedabfff640 1 -- 172.21.15.112:0/2807905806 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fedbc09bd50 con 0x7fedc4102230 2024-04-24T23:17:40.807 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.806+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fedac0a2b90 msgr2=0x7fedac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.806+0000 7fedc907d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fedac0a2b90 0x7fedac0a5050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fedc419b280 tx=0x7fedb4009680 comp rx=0 tx=0).stop 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.806+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4102230 msgr2=0x7fedc41a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.806+0000 7fedc907d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4102230 0x7fedc41a1320 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00e970 tx=0x7fedbc00ed70 comp rx=0 tx=0).stop 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.807+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 shutdown_connections 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.807+0000 7fedc907d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fedac0a2b90 0x7fedac0a5050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.807+0000 7fedc907d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedc4102230 0x7fedc41a1320 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.807+0000 7fedc907d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc4101870 0x7fedc419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.807+0000 7fedc907d640 1 --2- 172.21.15.112:0/2807905806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedc4100680 0x7fedc4199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.807+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 >> 172.21.15.112:0/2807905806 conn(0x7fedc40fbbf0 msgr2=0x7fedc4109480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.807+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 shutdown_connections 2024-04-24T23:17:40.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.807+0000 7fedc907d640 1 -- 172.21.15.112:0/2807905806 wait complete. 2024-04-24T23:17:40.820 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:17:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 -- 172.21.15.112:0/2014751303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c1033e0 msgr2=0x7fc53c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 --2- 172.21.15.112:0/2014751303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c1033e0 0x7fc53c109c70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc534009cd0 tx=0x7fc53402a8e0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 -- 172.21.15.112:0/2014751303 shutdown_connections 2024-04-24T23:17:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 --2- 172.21.15.112:0/2014751303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c1033e0 0x7fc53c109c70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 --2- 172.21.15.112:0/2014751303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53c102a20 0x7fc53c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 --2- 172.21.15.112:0/2014751303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c0ffa50 0x7fc53c0ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 -- 172.21.15.112:0/2014751303 >> 172.21.15.112:0/2014751303 conn(0x7fc53c0fb380 msgr2=0x7fc53c0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:40.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 -- 172.21.15.112:0/2014751303 shutdown_connections 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 -- 172.21.15.112:0/2014751303 wait complete. 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 Processor -- start 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.919+0000 7fc54355a640 1 -- start start 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc54355a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c0ffa50 0x7fc53c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc54355a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53c102a20 0x7fc53c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc54355a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c1033e0 0x7fc53c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc54355a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10d540 con 0x7fc53c0ffa50 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc54355a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10d3c0 con 0x7fc53c102a20 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc54355a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c10d6c0 con 0x7fc53c1033e0 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c0ffa50 0x7fc53c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c0ffa50 0x7fc53c197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54068/0 (socket says 172.21.15.112:54068) 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 -- 172.21.15.112:0/1959677802 learned_addr learned my addr 172.21.15.112:0/1959677802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc540af5640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53c102a20 0x7fc53c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc541af7640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c1033e0 0x7fc53c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 -- 172.21.15.112:0/1959677802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c1033e0 msgr2=0x7fc53c19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c1033e0 0x7fc53c19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 -- 172.21.15.112:0/1959677802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53c102a20 msgr2=0x7fc53c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53c102a20 0x7fc53c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 -- 172.21.15.112:0/1959677802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c19f710 con 0x7fc53c0ffa50 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc5412f6640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c0ffa50 0x7fc53c197af0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc53800c970 tx=0x7fc53800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc52e7fc640 1 -- 172.21.15.112:0/1959677802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538004220 con 0x7fc53c0ffa50 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c19fa00 con 0x7fc53c0ffa50 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc52e7fc640 1 -- 172.21.15.112:0/1959677802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fc5380043c0 con 0x7fc53c0ffa50 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c19ffa0 con 0x7fc53c0ffa50 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.920+0000 7fc52e7fc640 1 -- 172.21.15.112:0/1959677802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53801d060 con 0x7fc53c0ffa50 2024-04-24T23:17:40.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.921+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c0689b0 con 0x7fc53c0ffa50 2024-04-24T23:17:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.922+0000 7fc52e7fc640 1 -- 172.21.15.112:0/1959677802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc5380037a0 con 0x7fc53c0ffa50 2024-04-24T23:17:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.923+0000 7fc52e7fc640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5280a2b90 0x7fc5280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.923+0000 7fc540af5640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5280a2b90 0x7fc5280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.923+0000 7fc52e7fc640 1 -- 172.21.15.112:0/1959677802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fc5380cf3d0 con 0x7fc53c0ffa50 2024-04-24T23:17:40.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.923+0000 7fc540af5640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5280a2b90 0x7fc5280a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc53c199000 tx=0x7fc53000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:40.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:40.927+0000 7fc52e7fc640 1 -- 172.21.15.112:0/1959677802 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc538014030 con 0x7fc53c0ffa50 2024-04-24T23:17:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.108+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc53c02d1f0 con 0x7fc53c0ffa50 2024-04-24T23:17:41.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.109+0000 7fc52e7fc640 1 -- 172.21.15.112:0/1959677802 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fc538003e40 con 0x7fc53c0ffa50 2024-04-24T23:17:41.111 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:17:41.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5280a2b90 msgr2=0x7fc5280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:41.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5280a2b90 0x7fc5280a5050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc53c199000 tx=0x7fc53000b390 comp rx=0 tx=0).stop 2024-04-24T23:17:41.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c0ffa50 msgr2=0x7fc53c197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:41.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c0ffa50 0x7fc53c197af0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc53800c970 tx=0x7fc53800ce30 comp rx=0 tx=0).stop 2024-04-24T23:17:41.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 shutdown_connections 2024-04-24T23:17:41.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fc5280a2b90 0x7fc5280a5050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc53c1033e0 0x7fc53c19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc53c102a20 0x7fc53c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 --2- 172.21.15.112:0/1959677802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc53c0ffa50 0x7fc53c197af0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 >> 172.21.15.112:0/1959677802 conn(0x7fc53c0fb380 msgr2=0x7fc53c105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:41.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 shutdown_connections 2024-04-24T23:17:41.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.111+0000 7fc54355a640 1 -- 172.21.15.112:0/1959677802 wait complete. 2024-04-24T23:17:41.123 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1512502912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:17:41.123 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2763970798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:17:41.123 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:41 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2807905806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:17:41.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-24T23:17:41.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 -- 172.21.15.112:0/1516374826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe501074e0 msgr2=0x7fbe501098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:41.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 --2- 172.21.15.112:0/1516374826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe501074e0 0x7fbe501098d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbe48009ca0 tx=0x7fbe4802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 -- 172.21.15.112:0/1516374826 shutdown_connections 2024-04-24T23:17:41.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 --2- 172.21.15.112:0/1516374826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe501074e0 0x7fbe501098d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 --2- 172.21.15.112:0/1516374826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe50104bb0 0x7fbe50106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 --2- 172.21.15.112:0/1516374826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe50102280 0x7fbe50104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 -- 172.21.15.112:0/1516374826 >> 172.21.15.112:0/1516374826 conn(0x7fbe500fbbf0 msgr2=0x7fbe500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 -- 172.21.15.112:0/1516374826 shutdown_connections 2024-04-24T23:17:41.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.220+0000 7fbe55a53640 1 -- 172.21.15.112:0/1516374826 wait complete. 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe55a53640 1 Processor -- start 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe55a53640 1 -- start start 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe55a53640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe50102280 0x7fbe50197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe55a53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe50104bb0 0x7fbe50198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe55a53640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe501074e0 0x7fbe5019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe55a53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010d550 con 0x7fbe50104bb0 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe55a53640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010d3d0 con 0x7fbe501074e0 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe55a53640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5010d6d0 con 0x7fbe50102280 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe4e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe50104bb0 0x7fbe50198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:41.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe4e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe50104bb0 0x7fbe50198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54072/0 (socket says 172.21.15.112:54072) 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe4e7fc640 1 -- 172.21.15.112:0/2984342861 learned_addr learned my addr 172.21.15.112:0/2984342861 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe4f7fe640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe501074e0 0x7fbe5019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.221+0000 7fbe4effd640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe50102280 0x7fbe50197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe4e7fc640 1 -- 172.21.15.112:0/2984342861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe50102280 msgr2=0x7fbe50197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe4e7fc640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe50102280 0x7fbe50197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe4e7fc640 1 -- 172.21.15.112:0/2984342861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe501074e0 msgr2=0x7fbe5019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe4e7fc640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe501074e0 0x7fbe5019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe4e7fc640 1 -- 172.21.15.112:0/2984342861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5019f730 con 0x7fbe50104bb0 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe4e7fc640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe50104bb0 0x7fbe50198050 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400d740 tx=0x7fbe4400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe54a51640 1 -- 172.21.15.112:0/2984342861 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44017070 con 0x7fbe50104bb0 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5019fa20 con 0x7fbe50104bb0 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe54a51640 1 -- 172.21.15.112:0/2984342861 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fbe4400de50 con 0x7fbe50104bb0 2024-04-24T23:17:41.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe54a51640 1 -- 172.21.15.112:0/2984342861 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44010460 con 0x7fbe50104bb0 2024-04-24T23:17:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.222+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5019ff20 con 0x7fbe50104bb0 2024-04-24T23:17:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.223+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe500fd6e0 con 0x7fbe50104bb0 2024-04-24T23:17:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.224+0000 7fbe54a51640 1 -- 172.21.15.112:0/2984342861 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbe44004f10 con 0x7fbe50104bb0 2024-04-24T23:17:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.224+0000 7fbe54a51640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe380a2b90 0x7fbe380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.225+0000 7fbe4effd640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe380a2b90 0x7fbe380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.225+0000 7fbe54a51640 1 -- 172.21.15.112:0/2984342861 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fbe440ced50 con 0x7fbe50104bb0 2024-04-24T23:17:41.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.225+0000 7fbe4effd640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe380a2b90 0x7fbe380a5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009bd0 tx=0x7fbe40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:41.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.229+0000 7fbe54a51640 1 -- 172.21.15.112:0/2984342861 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbe440d2030 con 0x7fbe50104bb0 2024-04-24T23:17:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1512502912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:17:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2763970798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:17:41.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:41 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2807905806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:17:41.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.453+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fbe50062340 con 0x7fbe50104bb0 2024-04-24T23:17:41.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.461+0000 7fbe54a51640 1 -- 172.21.15.112:0/2984342861 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740377 (secure 0 0 0) 0x7fbe4409b270 con 0x7fbe50104bb0 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 18, 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: "active_gid": 14150, 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: "active_name": "x", 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addrs": { 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6800", 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2191305617 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6801", 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2191305617 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addr": "172.21.15.112:6801/2191305617", 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: "active_change": "2024-04-24T23:05:26.702200+0000", 2024-04-24T23:17:41.475 INFO:teuthology.orchestra.run.smithi112.stdout: "active_mgr_features": 4540701547738038271, 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "available": true, 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "standbys": [ 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24112, 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "z", 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:41.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.480 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.481 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-24T23:17:41.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.486 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-24T23:17:41.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:17:41.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-24T23:17:41.495 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-24T23:17:41.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-24T23:17:41.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-24T23:17:41.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.502 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.503 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:41.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-24T23:17:41.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-24T23:17:41.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.508 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-24T23:17:41.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-24T23:17:41.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.509 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.510 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:41.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:17:41.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.512 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.514 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.517 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-24T23:17:41.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-24T23:17:41.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-24T23:17:41.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-24T23:17:41.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.519 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-24T23:17:41.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.519 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.521 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.521 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.521 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:17:41.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:17:41.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.522 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-24T23:17:41.522 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-24T23:17:41.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:41.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:17:41.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:17:41.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.525 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.525 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-24T23:17:41.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-24T23:17:41.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.526 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-24T23:17:41.526 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-24T23:17:41.526 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-24T23:17:41.526 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.527 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:17:41.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.527 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-24T23:17:41.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-24T23:17:41.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-24T23:17:41.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:17:41.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:17:41.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:17:41.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:17:41.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:17:41.541 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:17:41.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-24T23:17:41.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-24T23:17:41.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-24T23:17:41.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.552 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-24T23:17:41.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:17:41.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.556 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.558 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.559 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-24T23:17:41.569 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.570 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.574 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.587 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.588 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-24T23:17:41.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.591 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.595 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.599 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.604 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.605 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:17:41.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:17:41.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.606 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.606 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.607 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:17:41.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:17:41.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.609 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:17:41.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:17:41.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:41.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.610 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.610 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.610 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.610 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.611 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:17:41.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:17:41.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.613 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-24T23:17:41.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:17:41.613 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.613 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.613 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:17:41.614 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:17:41.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:17:41.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:17:41.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.619 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:17:41.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.621 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.622 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.623 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:17:41.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:17:41.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:17:41.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.626 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.626 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.626 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.626 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.631 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.635 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.639 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-24T23:17:41.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.646 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.647 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.649 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.650 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.652 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.653 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.655 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.656 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.657 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.659 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.661 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.663 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.664 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.665 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.667 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-24T23:17:41.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-24T23:17:41.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:17:41.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.673 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.673 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.673 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-24T23:17:41.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-24T23:17:41.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:17:41.675 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.675 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.675 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:41.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:41.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:41.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:17:41.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.677 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-24T23:17:41.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-24T23:17:41.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.678 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-24T23:17:41.678 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:17:41.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.679 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.680 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.680 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-24T23:17:41.680 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.680 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.681 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.681 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.681 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.681 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.682 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.682 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.682 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.682 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.682 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.683 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.684 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.685 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.685 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.686 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.686 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.687 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.687 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.687 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.687 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.688 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.688 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.688 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.688 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.688 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.688 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.689 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.690 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.691 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.692 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-24T23:17:41.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-24T23:17:41.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.693 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-24T23:17:41.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.693 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.694 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.694 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.694 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-24T23:17:41.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-24T23:17:41.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-24T23:17:41.696 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.698 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.701 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.703 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.704 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-24T23:17:41.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-24T23:17:41.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-24T23:17:41.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-24T23:17:41.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-24T23:17:41.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.710 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.711 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.712 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.716 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.719 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.720 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:17:41.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.724 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.725 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.727 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.728 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.729 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.731 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.733 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.737 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.739 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.742 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.743 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.745 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.746 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.747 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.748 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.751 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.752 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.753 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.754 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.755 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.756 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.758 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.759 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.760 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.761 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.764 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.765 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.766 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.769 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.771 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.772 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.774 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.775 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.777 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.778 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.779 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-24T23:17:41.780 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-24T23:17:41.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-24T23:17:41.782 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.784 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.785 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.786 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.787 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.788 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.789 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-24T23:17:41.790 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-24T23:17:41.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.794 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.795 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.797 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.798 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.799 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.801 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.802 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-24T23:17:41.803 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-24T23:17:41.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.805 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.806 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.808 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.812 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.814 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.816 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.817 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.818 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.819 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.820 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.821 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.822 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.823 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.825 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.826 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.827 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.828 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.829 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.831 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-24T23:17:41.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.833 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.834 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.835 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.838 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.839 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.841 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.843 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-24T23:17:41.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-24T23:17:41.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-24T23:17:41.846 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.850 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.851 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.853 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.854 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.856 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.857 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.858 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.861 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.864 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.865 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.866 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-24T23:17:41.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.869 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.871 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.872 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.874 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.875 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.876 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-24T23:17:41.877 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-24T23:17:41.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-24T23:17:41.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-24T23:17:41.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.882 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.884 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-24T23:17:41.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.886 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.887 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.889 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.890 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.892 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.893 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.894 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.895 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.896 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.897 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.898 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.899 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.900 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.901 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.902 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.904 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-24T23:17:41.906 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-24T23:17:41.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-24T23:17:41.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:17:41.909 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.910 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.911 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.914 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.915 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.920 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.921 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24113, 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "y", 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:41.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.924 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.926 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.928 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.930 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.931 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.932 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.933 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.934 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.935 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.936 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-24T23:17:41.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-24T23:17:41.938 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:17:41.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-24T23:17:41.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.943 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.946 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.947 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.948 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.949 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.950 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-24T23:17:41.951 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:41.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-24T23:17:41.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:17:41.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:41.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.957 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.958 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.959 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.960 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.961 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.962 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.963 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:17:41.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.965 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.967 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-24T23:17:41.968 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:17:41.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:17:41.971 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.972 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.973 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.974 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.975 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.976 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:17:41.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.978 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:17:41.979 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:17:41.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.981 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.982 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.983 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:17:41.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.986 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.988 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.989 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.990 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:41.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.992 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.993 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.994 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.995 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.996 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:17:41.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:41.998 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:41.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.000 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.003 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.005 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.008 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.010 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:17:42.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:17:42.013 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.014 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.014 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.014 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-24T23:17:42.014 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:17:42.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.016 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-24T23:17:42.016 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:17:42.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.016 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.017 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.018 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.019 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.020 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.021 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.022 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.023 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.024 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.026 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.028 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.029 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.030 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.031 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.033 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.035 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.037 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.038 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.040 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.042 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.043 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.044 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.046 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.047 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.048 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.049 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.051 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.053 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.054 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.056 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.057 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.059 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.060 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:17:42.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:17:42.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:17:42.063 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.065 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.068 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.069 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.070 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.073 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.075 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.076 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-24T23:17:42.078 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.079 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.080 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.082 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.086 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.092 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.093 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.094 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:17:42.096 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.099 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.100 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.101 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.103 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.104 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-24T23:17:42.105 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:42.107 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.108 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.109 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.110 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.111 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.114 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.115 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.116 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-24T23:17:42.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:42.120 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.124 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.126 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.127 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-24T23:17:42.128 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.129 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.130 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.133 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.134 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.135 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.136 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.137 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.139 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.140 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.142 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.143 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:17:42.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.149 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.150 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.153 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.154 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.155 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.157 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.158 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.159 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.162 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.163 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.164 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.167 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.171 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.172 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.174 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-24T23:17:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-24T23:17:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-24T23:17:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-24T23:17:42.175 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.178 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.179 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.180 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.182 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.183 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.186 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.187 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.189 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.190 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.194 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-24T23:17:42.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-24T23:17:42.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-24T23:17:42.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:17:42.196 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.196 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.196 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.197 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-24T23:17:42.197 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.197 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.197 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.197 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-24T23:17:42.198 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:17:42.198 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.198 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.198 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.198 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.198 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.198 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.199 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:17:42.199 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.199 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-24T23:17:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-24T23:17:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.202 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.203 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.204 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.205 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:17:42.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.208 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.209 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.210 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.211 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.213 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.217 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.218 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.220 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-24T23:17:42.221 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-24T23:17:42.222 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.226 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.228 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.230 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-24T23:17:42.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.232 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.233 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.234 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.235 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.236 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.237 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.243 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.246 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.247 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-24T23:17:42.251 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.252 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.253 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-24T23:17:42.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-24T23:17:42.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-24T23:17:42.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.259 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.261 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.265 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-24T23:17:42.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.267 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.269 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.270 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.271 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.274 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.275 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.278 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.281 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.282 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.283 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.286 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.289 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.290 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.292 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-24T23:17:42.293 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-24T23:17:42.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.296 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.299 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.300 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-24T23:17:42.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-24T23:17:42.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.305 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.306 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.308 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.311 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.313 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.315 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.316 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.318 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.320 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.321 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.328 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.330 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.331 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.333 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.334 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.335 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: "modules": [ 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard", 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: "iostat", 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: "nfs", 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: "restful" 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.338 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.339 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.342 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.344 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-24T23:17:42.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-24T23:17:42.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.347 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-24T23:17:42.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.351 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-24T23:17:42.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.353 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-24T23:17:42.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:42.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-24T23:17:42.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.358 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:42.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.361 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.362 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.363 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.365 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.366 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.367 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.368 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.370 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.371 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-24T23:17:42.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.378 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.379 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.380 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-24T23:17:42.383 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-24T23:17:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-24T23:17:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.384 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.386 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-24T23:17:42.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-24T23:17:42.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.394 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.398 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-24T23:17:42.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-24T23:17:42.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-24T23:17:42.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.410 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-24T23:17:42.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.413 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.417 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.418 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.419 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-24T23:17:42.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-24T23:17:42.421 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-24T23:17:42.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-24T23:17:42.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.425 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.427 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.428 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-24T23:17:42.429 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.430 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.431 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-24T23:17:42.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.433 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-24T23:17:42.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.435 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-24T23:17:42.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-24T23:17:42.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.438 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.439 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.442 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.444 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.452 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.456 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-24T23:17:42.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-24T23:17:42.458 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-24T23:17:42.459 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.461 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.463 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.467 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.474 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.475 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-24T23:17:42.476 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-24T23:17:42.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-24T23:17:42.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-24T23:17:42.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.491 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.492 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.497 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:42.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.499 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-24T23:17:42.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-24T23:17:42.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-24T23:17:42.504 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.506 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.508 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-24T23:17:42.510 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-24T23:17:42.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-24T23:17:42.512 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-24T23:17:42.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-24T23:17:42.514 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.517 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.518 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.520 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.521 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-24T23:17:42.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.525 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-24T23:17:42.527 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.529 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.530 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.535 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-24T23:17:42.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-24T23:17:42.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-24T23:17:42.539 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.541 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.545 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.546 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.547 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.548 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.550 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.556 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.557 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.558 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.560 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.561 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.562 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.569 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.570 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.572 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.575 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.577 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.579 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.580 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.583 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.584 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-24T23:17:42.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.587 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.589 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.592 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.593 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.594 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.595 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.596 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.599 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.605 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.606 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.607 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-24T23:17:42.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.614 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.615 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.618 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.619 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.620 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.622 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.623 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.624 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-24T23:17:42.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-24T23:17:42.630 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.632 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.636 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.641 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.642 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.644 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.645 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.646 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.648 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.650 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.651 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.652 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.653 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.656 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.661 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.662 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.663 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.664 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.665 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.666 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-24T23:17:42.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-24T23:17:42.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-24T23:17:42.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.672 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-24T23:17:42.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.676 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.680 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.683 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.684 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.685 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.687 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.688 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.689 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.690 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.691 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.693 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.694 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-24T23:17:42.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.698 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.701 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.703 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.704 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.705 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-24T23:17:42.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.707 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.708 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-24T23:17:42.709 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.712 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.714 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.715 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.716 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.717 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.718 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-24T23:17:42.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-24T23:17:42.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-24T23:17:42.721 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.722 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.723 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.727 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.728 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.732 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.735 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.737 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.740 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-24T23:17:42.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-24T23:17:42.742 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.743 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-24T23:17:42.746 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "services": { 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard": "https://172.21.15.112:8443/" 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "always_on_modules": { 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus": [ 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-24T23:17:42.747 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific": [ 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-24T23:17:42.748 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy": [ 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-24T23:17:42.749 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "reef": [ 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "squid": [ 2024-04-24T23:17:42.750 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "last_failure_osd_epoch": 3, 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "active_clients": [ 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 182592862 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "libcephsqlite", 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-24T23:17:42.752 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 4077926577 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1978319668 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 845221192 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-24T23:17:42.754 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:42.756 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:42 smithi150 ceph-mon[32247]: pgmap v451: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 KiB/s wr, 18 op/s 2024-04-24T23:17:42.756 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1959677802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:17:42.756 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:42 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2984342861' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:17:42.756 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1512502912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:17:42.756 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2763970798' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:17:42.756 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:41 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2807905806' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:17:42.756 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:42 smithi187 ceph-mon[33292]: pgmap v451: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 KiB/s wr, 18 op/s 2024-04-24T23:17:42.756 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1959677802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:17:42.757 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:42 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2984342861' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:17:42.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe380a2b90 msgr2=0x7fbe380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:42.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe380a2b90 0x7fbe380a5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009bd0 tx=0x7fbe40005f90 comp rx=0 tx=0).stop 2024-04-24T23:17:42.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe50104bb0 msgr2=0x7fbe50198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:42.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe50104bb0 0x7fbe50198050 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400d740 tx=0x7fbe4400dc00 comp rx=0 tx=0).stop 2024-04-24T23:17:42.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 shutdown_connections 2024-04-24T23:17:42.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fbe380a2b90 0x7fbe380a5050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbe501074e0 0x7fbe5019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe50104bb0 0x7fbe50198050 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 --2- 172.21.15.112:0/2984342861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe50102280 0x7fbe50197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 >> 172.21.15.112:0/2984342861 conn(0x7fbe500fbbf0 msgr2=0x7fbe50102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:42.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 shutdown_connections 2024-04-24T23:17:42.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:17:41.467+0000 7fbe55a53640 1 -- 172.21.15.112:0/2984342861 wait complete. 2024-04-24T23:17:42.764 DEBUG:tasks.check_counter:Getting stats from a 2024-04-24T23:17:42.764 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-24T23:17:42.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:42 smithi112 ceph-mon[29186]: pgmap v451: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.7 KiB/s wr, 18 op/s 2024-04-24T23:17:42.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1959677802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:17:42.764 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:42 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2984342861' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-24T23:17:43.053 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:17:44.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:44 smithi150 ceph-mon[32247]: pgmap v452: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-24T23:17:44.385 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:44 smithi112 ceph-mon[29186]: pgmap v452: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-24T23:17:44.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:44 smithi187 ceph-mon[33292]: pgmap v452: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 431, 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 417, 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 638936, 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 157094, 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-24T23:17:45.030 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.044317764, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.015645143, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.017072496, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.010290687, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 416, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.015791476, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000037960 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.031 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 534944, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 56288 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 8, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 74, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 4542, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 194190, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-04-24T23:17:45.032 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.005790952, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.003328283, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001885760, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000047206, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 73, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.005523777, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000075668 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 193168 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 456, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 458, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 5403599, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 145009, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.033334447, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.008316153, 2024-04-24T23:17:45.034 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.012107018, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.010574102, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 456, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.013812352, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000030290 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:17:45.035 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1365, 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.032156749, 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000023558 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-24T23:17:45.036 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.037 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-24T23:17:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-24T23:17:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-24T23:17:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-24T23:17:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-04-24T23:17:45.038 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 968, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 968, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 968, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 1, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-04-24T23:17:45.039 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-24T23:17:45.040 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:17:45.041 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:17:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:17:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-04-24T23:17:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:17:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-04-24T23:17:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-04-24T23:17:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.042 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-24T23:17:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-24T23:17:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-24T23:17:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-24T23:17:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-24T23:17:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-24T23:17:45.046 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-24T23:17:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-24T23:17:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:17:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-04-24T23:17:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-04-24T23:17:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-24T23:17:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-24T23:17:45.047 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-24T23:17:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:17:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:17:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-24T23:17:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-24T23:17:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:17:45.048 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:17:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:17:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-24T23:17:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-24T23:17:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:17:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-24T23:17:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-24T23:17:45.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-24T23:17:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-24T23:17:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-04-24T23:17:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-24T23:17:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-24T23:17:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 2817, 2024-04-24T23:17:45.050 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-24T23:17:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-24T23:17:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-04-24T23:17:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-24T23:17:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-24T23:17:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 5, 2024-04-24T23:17:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-04-24T23:17:45.051 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 9481014, 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 9479876, 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 2817 2024-04-24T23:17:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-24T23:17:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 7, 2024-04-24T23:17:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 1023, 2024-04-24T23:17:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 1016, 2024-04-24T23:17:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 5, 2024-04-24T23:17:45.054 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 121, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 116, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 968, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 1993, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 1025, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 51256, 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215420 2024-04-24T23:17:45.055 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-24T23:17:45.057 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.058 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.059 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.060 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-24T23:17:45.061 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-24T23:17:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.062 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.063 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.064 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.065 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.067 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.068 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-24T23:17:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-24T23:17:45.069 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:17:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:17:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-24T23:17:45.070 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 1, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 0, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1295, 2024-04-24T23:17:45.071 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:17:45.072 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:17:45.073 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 205399, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 56, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 9176, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 176, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 703928, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 3934, 2024-04-24T23:17:45.075 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 776, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 21, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 3, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 776, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 776, 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.784979140, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001011571 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 776, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 777 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 776, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 776, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 458, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 316, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 1, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-04-24T23:17:45.078 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-24T23:17:45.079 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 94, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 4, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 7, 2024-04-24T23:17:45.080 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 7, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-04-24T23:17:45.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:45.082 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 891, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 5958113, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 891, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 891, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 5958113, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-24T23:17:45.083 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 776, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 5286715, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 776, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 5286715, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.084 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 776, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 776, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 776, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 776, 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:45.085 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-24T23:17:45.086 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x563373e626e0": { 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:45.087 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:45.088 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:45.908 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-24T23:17:45.908 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-24T23:17:45.908 DEBUG:tasks.check_counter:Getting stats from d 2024-04-24T23:17:45.908 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-24T23:17:46.158 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:17:46.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:46 smithi112 ceph-mon[29186]: pgmap v453: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-24T23:17:46.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:46 smithi150 ceph-mon[32247]: pgmap v453: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-24T23:17:46.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:46 smithi187 ceph-mon[33292]: pgmap v453: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1023 B/s wr, 11 op/s 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-24T23:17:47.921 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001070724, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000193912, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:47.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 100, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 97, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 443776, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 58592, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.014693860, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.005163139, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.009830738, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.001702730, 2024-04-24T23:17:47.923 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 96, 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004257121, 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000044345 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 443776, 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 58592 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:17:47.924 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 5, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 75, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3152, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 90864, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.006211214, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.003954691, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001615583, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 74, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004243760, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000057348 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.925 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 90864 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-24T23:17:47.926 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:17:47.927 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-24T23:17:47.928 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148880, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 29, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-24T23:17:47.929 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:47.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 101, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 429679, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 101, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 101, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 429679, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:47.931 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:48.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:48 smithi112 ceph-mon[29186]: pgmap v454: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-24T23:17:48.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:48 smithi150 ceph-mon[32247]: pgmap v454: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-24T23:17:48.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:48 smithi187 ceph-mon[33292]: pgmap v454: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-24T23:17:48.612 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-24T23:17:48.612 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-24T23:17:48.612 DEBUG:tasks.check_counter:Getting stats from g 2024-04-24T23:17:48.612 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-24T23:17:48.852 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:17:50.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:50 smithi112 ceph-mon[29186]: pgmap v455: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T23:17:50.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:50 smithi150 ceph-mon[32247]: pgmap v455: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T23:17:50.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:50 smithi187 ceph-mon[33292]: pgmap v455: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T23:17:50.606 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:50.606 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:17:50.606 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 295, 2024-04-24T23:17:50.606 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 297, 2024-04-24T23:17:50.606 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 99065, 2024-04-24T23:17:50.606 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 93811, 2024-04-24T23:17:50.606 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-24T23:17:50.606 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.030400904, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.012342442, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.006723286, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.007898363, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 295, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.012693747, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000043029 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 504, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 488, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 6977357, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 180185, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.043821120, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.012383212, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.022292706, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.009628329, 2024-04-24T23:17:50.608 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 488, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.013989999, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000028668 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 538480, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 58016 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-04-24T23:17:50.609 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 73, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1508, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 195477, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 5, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 4, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.006374835, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.003705225, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001987687, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000052253, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 71, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004618957, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000065055 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.610 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 194512 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 1, 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-MDSRank": { 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-24T23:17:50.611 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 1289, 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.029537976, 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000022915 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "finisher-PurgeQueue": { 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "queue_len": 0, 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "complete_latency": { 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": { 2024-04-24T23:17:50.612 INFO:teuthology.orchestra.run.smithi112.stdout: "request": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "reply": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "reply_latency": { 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_reply": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "forward": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_complete": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_fetch_keys": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_commit": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_split": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_merge": 0, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes": 1231, 2024-04-24T23:17:50.613 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_top": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_bottom": 1231, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pin_tail": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_pinned": 1231, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_expired": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "inodes_with_caps": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "subtrees": 1, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_hit": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_forward": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_discover": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_remote_ino": 0, 2024-04-24T23:17:50.614 INFO:teuthology.orchestra.run.smithi112.stdout: "traverse_lock": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "load_cent": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "q": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "exported": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "exported_inodes": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "imported": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "imported_inodes": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_dir_fetch": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "openino_peer_discover": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rfiles": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rbytes": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "root_rsnaps": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_set_tag": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:17:50.615 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "scrub_file_inodes": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps": 0, 2024-04-24T23:17:50.616 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_cap_release": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "process_request_cap_release": 0 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_cache": { 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_update_receipt": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_try_discover": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_send_discover": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "dir_handle_discover": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_delayed": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_created": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_enqueued": 0, 2024-04-24T23:17:50.617 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_reintegrated": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "strays_migrated": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_processing": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_started": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "recovery_completed": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_exportdir": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_flush": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_fragstats": 0, 2024-04-24T23:17:50.618 INFO:teuthology.orchestra.run.smithi112.stdout: "ireq_inodestats": 0 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_log": { 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "evlrg": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "evadd": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "evex": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "evtrm": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "ev": 3467, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "evexg": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "evexd": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "segadd": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "segex": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "segtrm": 0, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "seg": 6, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "segmjr": 2, 2024-04-24T23:17:50.619 INFO:teuthology.orchestra.run.smithi112.stdout: "segexg": 0, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "segexd": 0, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "expos": 4194304, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "wrpos": 10533821, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "rdpos": 10532683, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "jlat": { 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "replayed": 3467 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_mem": { 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "ino": 7, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "ino+": 1287, 2024-04-24T23:17:50.620 INFO:teuthology.orchestra.run.smithi112.stdout: "ino-": 1280, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "dir": 5, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "dir+": 132, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "dir-": 127, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "dn": 1231, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "dn+": 2518, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "dn-": 1287, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "cap": 0, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "cap+": 0, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "cap-": 0, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "rss": 55952, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "heap": 215420 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_server": { 2024-04-24T23:17:50.621 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_client_request": 0, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "dispatch_server_request": 0, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_request": 0, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_client_session": 0, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "handle_peer_request": 0, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "req_create_latency": { 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getattr_latency": { 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.622 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getfilelock_latency": { 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "req_link_latency": { 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookup_latency": { 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.623 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookuphash_latency": { 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupino_latency": { 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupname_latency": { 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.624 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupparent_latency": { 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lookupsnap_latency": { 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "req_lssnap_latency": { 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.625 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mkdir_latency": { 2024-04-24T23:17:50.626 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.626 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.626 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mknod_latency": { 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "req_mksnap_latency": { 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "req_open_latency": { 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.627 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "req_readdir_latency": { 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rename_latency": { 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "req_renamesnap_latency": { 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "req_snapdiff_latency": { 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmdir_latency": { 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmsnap_latency": { 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.629 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "req_rmxattr_latency": { 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setattr_latency": { 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setdirlayout_latency": { 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.630 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setfilelock_latency": { 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setlayout_latency": { 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "req_setxattr_latency": { 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.631 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "req_symlink_latency": { 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "req_unlink_latency": { 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:17:50.632 INFO:teuthology.orchestra.run.smithi112.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "req_getvxattr_latency": { 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_sessions": { 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "session_count": 1, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "session_add": 1, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "session_remove": 0, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_open": 1, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "sessions_stale": 0, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "total_load": 0, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "average_load": 0, 2024-04-24T23:17:50.633 INFO:teuthology.orchestra.run.smithi112.stdout: "avg_session_uptime": 1298, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "mdthresh_evicted": 0 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:17:50.634 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:17:50.635 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:17:50.636 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 197399, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 54, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 9176, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 176, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-24T23:17:50.637 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 889080, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 4986, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "objecter": { 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "op_active": 0, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "op_laggy": 0, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send": 682, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "op_send_bytes": 21, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "op_resend": 3, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "op_reply": 682, 2024-04-24T23:17:50.638 INFO:teuthology.orchestra.run.smithi112.stdout: "op_latency": { 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 682, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.850977388, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.001247767 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "op_inflight": 0, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "oplen_avg": { 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 682, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 683 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "op": 682, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "op_r": 682, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "op_w": 0, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "op_rmw": 0, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "op_pg": 0, 2024-04-24T23:17:50.639 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_stat": 357, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_create": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_read": 323, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_write": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writefull": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_writesame": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_append": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_zero": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_truncate": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_delete": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_mapext": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_sparse_read": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_clonerange": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_getxattr": 1, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_setxattr": 0, 2024-04-24T23:17:50.640 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_rmxattr": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_call": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_watch": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_notify": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "osdop_other": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_active": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_send": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_resend": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "linger_ping": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_active": 0, 2024-04-24T23:17:50.641 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_send": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "poolop_resend": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_active": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_send": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "poolstat_resend": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_active": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_send": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "statfs_resend": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "command_active": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "command_send": 0, 2024-04-24T23:17:50.642 INFO:teuthology.orchestra.run.smithi112.stdout: "command_resend": 0, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "map_epoch": 94, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "map_full": 0, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "map_inc": 4, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_sessions": 6, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_open": 6, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_session_close": 0, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_laggy": 0, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_wr": 0, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_rd": 2, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_del": 0 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "oft": { 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_objs": 0, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:17:50.643 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_updates": 0, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "omap_total_removes": 0 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "osd-slow-ops": { 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "slow_ops_count": 0 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "purge_queue": { 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops": 0, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing": 0, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executing_high_water": 0, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed_ops": 0, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_executed": 0, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "pq_item_in_journal": 0 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.644 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:17:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-24T23:17:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 799, 2024-04-24T23:17:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 6997142, 2024-04-24T23:17:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 799, 2024-04-24T23:17:50.645 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 799, 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 6997142, 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.646 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_bytes": { 2024-04-24T23:17:50.647 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:50.647 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-24T23:17:50.647 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:50.647 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-24T23:17:50.647 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-24T23:17:50.647 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 682, 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 6339522, 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 682, 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 6339522, 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-objecter_ops": { 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 1024, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 682, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 682, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 682, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 682, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.649 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-24T23:17:50.650 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-write_buf_throttle-0x559092bb66e0": { 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 3758096384, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:50.651 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 0, 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 0, 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:50.652 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:51.435 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-24T23:17:51.435 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-24T23:17:51.435 DEBUG:tasks.check_counter:Getting stats from j 2024-04-24T23:17:51.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-24T23:17:51.678 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.a/config 2024-04-24T23:17:52.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:52 smithi150 ceph-mon[32247]: pgmap v456: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T23:17:52.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:52 smithi187 ceph-mon[33292]: pgmap v456: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T23:17:52.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:52 smithi112 ceph-mon[29186]: pgmap v456: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 10 op/s 2024-04-24T23:17:53.453 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-24T23:17:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:17:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 102, 2024-04-24T23:17:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 99, 2024-04-24T23:17:53.453 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 444320, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 59872, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.017669430, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.006227237, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.010715127, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.001828323, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 98, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.004424767, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000045150 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.454 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 444320, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 59872 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 5, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 77, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3152, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 91216, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 3, 2024-04-24T23:17:53.455 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 2, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.005876279, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.003804383, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.001426327, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 76, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.003794608, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000049929 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.456 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 91216 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.000319946, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000265514, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-24T23:17:53.457 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:17:53.458 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:17:53.459 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:17:53.460 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:17:53.461 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148823, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 27, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-24T23:17:53.462 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-24T23:17:53.463 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 103, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 429939, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 103, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 103, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 429939, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-24T23:17:53.464 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-24T23:17:53.465 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-24T23:17:53.465 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:53.465 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-24T23:17:53.465 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-24T23:17:54.181 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:54 smithi112 ceph-mon[29186]: pgmap v457: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T23:17:54.212 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-24T23:17:54.212 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-24T23:17:54.212 DEBUG:tasks.check_counter:Getting stats from b 2024-04-24T23:17:54.212 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-24T23:17:54.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:54 smithi150 ceph-mon[32247]: pgmap v457: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T23:17:54.463 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:17:54.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:54 smithi187 ceph-mon[33292]: pgmap v457: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-24T23:17:56.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:56 smithi150 ceph-mon[32247]: pgmap v458: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 11797, 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 9597, 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3417415, 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 25435285, 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 12, 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-04-24T23:17:56.355 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.723342047, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.285856566, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.312491016, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.150283636, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 9595, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.056467552, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000110106 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 9571, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001335907, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000139 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.356 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 37541, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 72706, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 12943622, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 60619571, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 15, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 5, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 2.258704013, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 1.002407158, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 1.873356953, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.205130446, 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:56.357 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 72705, 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2.281574904, 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000031381 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 89405, 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.012810859, 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000143 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 631552, 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 100880 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:17:56.358 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 10105, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 8799, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 3007548, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 24607014, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 51, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 10, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.732682917, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.321742485, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.315690294, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.142489491, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8798, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.543908009, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000061821 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.359 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8558, 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001334887, 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000155 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 206096 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-24T23:17:56.360 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5145, 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 21.460376034, 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.004171112 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 9569, 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.810199149, 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000189173 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-24T23:17:56.361 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 16983, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 16937, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 16937, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 34.085083666, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002012462 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 46, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 139, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 14, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 4, 2024-04-24T23:17:56.362 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 929, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 584, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 323, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 22, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 63, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 5747, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 2, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 2, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 6, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 17442, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 17368, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-24T23:17:56.363 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 20, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 45, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 5552, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 51, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 148, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 5678, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 4, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 234, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 5678, 2024-04-24T23:17:56.364 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 530, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 5, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 525, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 5148, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 81, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 1639, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 16310, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 13144, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 9355, 2024-04-24T23:17:56.365 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 115, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 6774 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 45, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 9, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 5149, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 5140, 2024-04-24T23:17:56.366 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 5, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 45, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 18, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-24T23:17:56.367 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 34209, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 34209, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 42, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 42, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 4, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-24T23:17:56.368 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 74964871, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1126, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 49.814232469, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.044239993 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 43, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 10740, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 10697, 2024-04-24T23:17:56.369 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 49, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 770, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 721, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 894, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 13565, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 12671, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 2, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 10715, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 10713, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 96504, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 231804 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 19043, 2024-04-24T23:17:56.370 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 2070, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 16983, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 58, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 4223, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 10088, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 18.229013446, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001806999 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 66, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.053451903, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000809877 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-24T23:17:56.371 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 321, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.318148370, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000991116 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-24T23:17:56.372 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000202892, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000202892 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-24T23:17:56.373 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.374 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.374 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.374 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-24T23:17:56.374 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.374 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.374 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.374 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 623, 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.886993081, 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001423744 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-24T23:17:56.375 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000277549, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000277549 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-24T23:17:56.376 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 675, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.147955598, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001700674 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001982584, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000991292 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.377 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 315, 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 4.746178127, 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.015067232 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.379 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.765228730, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.255076243 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.380 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.381 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8, 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.019669009, 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002458626 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-24T23:17:56.382 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4834, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 7.915982377, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001637563 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-04-24T23:17:56.383 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 3, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 2, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 10430, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 10430, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 183, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:17:56.384 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:17:56.385 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:17:56.386 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 320866, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 134, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 528, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 6, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 9176, 2024-04-24T23:17:56.387 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 176, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 873005, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 12043, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-24T23:17:56.388 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 17912, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 74967949, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 37, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 17912, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 17912, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 64.002732868, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003573176 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 17912, 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 18166 2024-04-24T23:17:56.389 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 17912, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 5783, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 12129, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 11, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 100, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 1221, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 74, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-24T23:17:56.390 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 14, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 10596, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 5683, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 22, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:17:56.391 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-24T23:17:56.392 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 94, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 9, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 393, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 52 2024-04-24T23:17:56.393 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 41, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-24T23:17:56.394 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 7, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 10040, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 5140, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 59439, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 14002192, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 59439, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:17:56.395 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 59439, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 14002192, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:17:56.396 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 17912, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 75525777, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 17912, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 75525777, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:56.397 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 17912, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 17912, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 17912, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 17912, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.398 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 5140, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 557828, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 5140, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 95, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 557828, 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:56.399 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55570a862500": { 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 34209, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 70770567, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 34209, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:17:56.400 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 1126, 2024-04-24T23:17:56.401 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 70770567, 2024-04-24T23:17:56.401 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:56.401 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:56.401 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:56.401 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:56.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.401 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:56.401 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-24T23:17:56.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:56 smithi187 ceph-mon[33292]: pgmap v458: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:17:56.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:56 smithi112 ceph-mon[29186]: pgmap v458: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-24T23:17:57.218 INFO:tasks.check_counter:Daemon mds.b mds.exported=45 2024-04-24T23:17:57.218 INFO:tasks.check_counter:Daemon mds.b mds.imported=51 2024-04-24T23:17:57.218 DEBUG:tasks.check_counter:Getting stats from e 2024-04-24T23:17:57.219 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-24T23:17:57.467 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:17:58.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:17:58 smithi150 ceph-mon[32247]: pgmap v459: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T23:17:58.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:17:58 smithi187 ceph-mon[33292]: pgmap v459: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T23:17:58.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:17:58 smithi112 ceph-mon[29186]: pgmap v459: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-24T23:17:59.152 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 162, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 143, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 571742, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 72544, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 6, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 2, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.023078678, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.007050047, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.014044037, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.003537211, 2024-04-24T23:17:59.153 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 142, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.008392941, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000059105 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 562544, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 61408 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 71, 2024-04-24T23:17:59.154 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 143, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 4214194, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 220065, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.012415002, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.004252576, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.005378438, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.001569248, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 142, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.008025565, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000056518 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.155 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 199072 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 623, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 625, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 5522816, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 198172, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 5, 2024-04-24T23:17:59.156 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 4, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.061759654, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.020581882, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.017972927, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.018131331, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 623, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.024930934, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000040017 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.157 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1353, 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.037316882, 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000027580 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.158 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 0, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 0, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.159 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 1543, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 1543, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 1543, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-04-24T23:17:59.160 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 1, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 0, 2024-04-24T23:17:59.161 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:17:59.162 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 0 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-24T23:17:59.163 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:17:59.164 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 4797, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-24T23:17:59.165 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 0, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 8, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 13725897, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 13724759, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.166 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 4797 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 9, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 1577, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 1568, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 7, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 164, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 157, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 1543, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 3111, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 1568, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-04-24T23:17:59.167 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 0, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 0, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 56308, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 215420 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 0, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 0, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 0, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 0, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 0, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.168 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-24T23:17:59.169 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.170 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-24T23:17:59.171 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-24T23:17:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.172 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.173 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.174 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.175 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.176 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.177 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-24T23:17:59.178 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 1, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 0, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 0, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 1284, 2024-04-24T23:17:59.179 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:17:59.180 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:17:59.181 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 205399, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 56, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-24T23:17:59.182 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 9176, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 176, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 1115704, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 6256, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-24T23:17:59.183 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 724, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 21, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 3, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 724, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 724, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.396992140, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000548331 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.184 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 724, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 725 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 724, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 724, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 0, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 387, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 0, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 335, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 0, 2024-04-24T23:17:59.185 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 1, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-24T23:17:59.186 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-24T23:17:59.187 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 94, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 4, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 7, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 7, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 0, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 2, 2024-04-24T23:17:59.188 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 0 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 0, 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:17:59.189 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 0, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 0, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 0, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 852, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 10222655, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 852, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:17:59.190 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 852, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 10222655, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:59.191 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 724, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 9531598, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 724, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 9531598, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:59.192 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 724, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 724, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 724, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 724, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:59.193 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.194 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55d5eeb2a6e0": { 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 0, 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:17:59.195 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:17:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:17:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:17:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.196 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:17:59.196 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-24T23:17:59.960 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-24T23:17:59.960 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-24T23:17:59.960 DEBUG:tasks.check_counter:Getting stats from h 2024-04-24T23:17:59.960 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-24T23:18:00.206 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:18:00.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:00 smithi150 ceph-mon[32247]: pgmap v460: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:00.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:00 smithi187 ceph-mon[33292]: pgmap v460: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:00.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:00 smithi112 ceph-mon[29186]: pgmap v460: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:01.916 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-24T23:18:01.916 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:18:01.916 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 2814, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 3962, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2462361, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 5679046, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 10, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.245107839, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.109072384, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.098842900, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.033914212, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3967, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.224296196, 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000056540 2024-04-24T23:18:01.917 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4179, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000679590, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000162 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1741, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1695, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1020126, 2024-04-24T23:18:01.918 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 4930333, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 9, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 7, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.134005747, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.041457903, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.046424049, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.039562589, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1694, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.133525061, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000078822 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 196, 2024-04-24T23:18:01.919 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000030281, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000154 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 570000, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 66752 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3801, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 8684, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 1373340, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 5744259, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 25, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 8, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.300802165, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.133475716, 2024-04-24T23:18:01.920 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.167316553, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.022805892, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8676, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.514672559, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000059321 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 8628, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.002325660, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000269 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 200128 2024-04-24T23:18:01.921 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 432, 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2.044688946, 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.004733076 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-24T23:18:01.922 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2785, 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.349422477, 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000125465 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 1877, 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 1877, 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1877, 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 4.103388161, 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002186141 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.923 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 11, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 16, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 1, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 2, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 13, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 13, 2024-04-24T23:18:01.924 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 63, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 0, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 0, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 4, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 2544, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 2106, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 5, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 7, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 36, 2024-04-24T23:18:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 7, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 1537, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 155, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 2794, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 107894170, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 155, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 19, 2024-04-24T23:18:01.926 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 19, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 136, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 209, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 1740, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 1069, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 863, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 13, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 1876 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-24T23:18:01.927 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 5, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 1, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 1537, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 1537, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:18:01.928 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 7, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 4797, 2024-04-24T23:18:01.929 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 4797, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 8, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 8, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 13725897, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-04-24T23:18:01.930 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 144, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.341149466, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002369093 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 21, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 1574, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 1553, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 20, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 161, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 141, 2024-04-24T23:18:01.931 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 16, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 3117, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 3101, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 0, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 1542, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 1542, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 66640, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 223612 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 2829, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 5, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 1877, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 2, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 344, 2024-04-24T23:18:01.932 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.933 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 224, 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.070516012, 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000314803 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.934 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.935 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-24T23:18:01.936 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.936 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.936 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.936 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.936 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-24T23:18:01.936 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.937 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 116, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.110809198, 2024-04-24T23:18:01.938 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000955251 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.939 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 91, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.872302524, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.020574753 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.940 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.941 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-24T23:18:01.942 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1446, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2.049760427, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001417538 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.943 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 1, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 1585, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 1585, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 1288, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-24T23:18:01.944 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:18:01.945 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:18:01.946 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 281474, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 105, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:18:01.947 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 9176, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 176, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 85125, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 406, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-24T23:18:01.948 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 3471, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 9888387, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 33, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 3471, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3471, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 11.758767343, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003387717 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-24T23:18:01.949 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 3471, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 3528 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 3471, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 194, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 3277, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 10, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 39, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 183, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 25, 2024-04-24T23:18:01.950 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 3021, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 155, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 20, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-24T23:18:01.951 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-24T23:18:01.952 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 94, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 4, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-24T23:18:01.953 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 57, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 18 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-24T23:18:01.954 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 17, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 9, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 3007, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 1537, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 8352, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 4093266, 2024-04-24T23:18:01.955 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 8352, 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 8352, 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 4093266, 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.956 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 3471, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 10055216, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 3471, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 10055216, 2024-04-24T23:18:01.957 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:18:01.958 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 3471, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 3471, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 3471, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 3471, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:01.959 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 1537, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 166829, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 1537, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 39, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 166829, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.960 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x559956db0500": { 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 4797, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 9531593, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 4797, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 144, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 9531593, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:01.961 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:01.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.962 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:01.962 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-24T23:18:02.318 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:02 smithi150 ceph-mon[32247]: pgmap v461: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:02.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:02 smithi187 ceph-mon[33292]: pgmap v461: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:02.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:02 smithi112 ceph-mon[29186]: pgmap v461: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:02.724 INFO:tasks.check_counter:Daemon mds.h mds.exported=7 2024-04-24T23:18:02.724 INFO:tasks.check_counter:Daemon mds.h mds.imported=7 2024-04-24T23:18:02.725 DEBUG:tasks.check_counter:Getting stats from k 2024-04-24T23:18:02.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-24T23:18:02.965 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.b/config 2024-04-24T23:18:04.401 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:04 smithi150 ceph-mon[32247]: pgmap v462: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:18:04.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:04 smithi187 ceph-mon[33292]: pgmap v462: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:18:04.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:04 smithi112 ceph-mon[29186]: pgmap v462: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:18:04.658 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-04-24T23:18:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:18:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 3105, 2024-04-24T23:18:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 4315, 2024-04-24T23:18:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 2481933, 2024-04-24T23:18:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 5519175, 2024-04-24T23:18:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 9, 2024-04-24T23:18:04.658 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 3, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.273472122, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.127196325, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.136733756, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.023314921, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 4320, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.207659095, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000048069 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5721, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.001041376, 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000182 2024-04-24T23:18:04.659 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 565296, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 70432 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 1087, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 1163, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 420662, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 1806483, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 16, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 6, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.098838311, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.037093919, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.025260560, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.026828694, 2024-04-24T23:18:04.660 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1162, 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.123053121, 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000105897 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 37, 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000022355, 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000604 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 201248 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:18:04.661 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_messages": 2839, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages": 5838, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_bytes": 896277, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_bytes": 3061015, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_created_connections": 19, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_active_connections": 8, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_total_time": 0.204711346, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_send_time": 0.084616338, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_recv_time": 0.107138492, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_running_fast_dispatch_time": 0.022187627, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5830, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.260013401, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000044599 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.662 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 5872, 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000913255, 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000155 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "cct": { 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "total_workers": 1, 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "unhealthy_workers": 0 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-MDSRank": { 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-24T23:18:04.663 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 502, 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.023636294, 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002039116 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "finisher-PurgeQueue": { 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "queue_len": 0, 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "complete_latency": { 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1763, 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.175162061, 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000099354 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "mds": { 2024-04-24T23:18:04.664 INFO:teuthology.orchestra.run.smithi150.stdout: "request": 1588, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "reply": 1588, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "reply_latency": { 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1588, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2.491684395, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001569070 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_reply": 0, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "forward": 0, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_complete": 0, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_fetch_keys": 0, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_commit": 11, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_split": 0, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_merge": 0, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes": 18, 2024-04-24T23:18:04.665 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_top": 1, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_bottom": 7, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pin_tail": 10, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_pinned": 13, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_expired": 142, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "inodes_with_caps": 1, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "caps": 1, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "subtrees": 3, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse": 2956, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_hit": 2177, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_forward": 0, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_discover": 18, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_remote_ino": 0, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "traverse_lock": 0, 2024-04-24T23:18:04.666 INFO:teuthology.orchestra.run.smithi150.stdout: "load_cent": 0, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "q": 0, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "exported": 18, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "exported_inodes": 50, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "imported": 18, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "imported_inodes": 964, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_dir_fetch": 0, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_backtrace_fetch": 169, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "openino_peer_discover": 0, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rfiles": 2794, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rbytes": 107894170, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "root_rsnaps": 0, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_fetch": 169, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_set_tag": 0, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:18:04.667 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_inodes": 5, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_dirfrag_rstats": 5, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "scrub_file_inodes": 164, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_revoke": 235, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_grant": 1068, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps": 507, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_caps_dirty": 277, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_cap_release": 11, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "process_request_cap_release": 1586 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_cache": { 2024-04-24T23:18:04.668 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_update_receipt": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_try_discover": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_send_discover": 18, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "dir_handle_discover": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_delayed": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_created": 964, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_enqueued": 964, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_reintegrated": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "strays_migrated": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_processing": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:18:04.669 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_started": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "recovery_completed": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_exportdir": 18, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_flush": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_fragstats": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "ireq_inodestats": 0 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_log": { 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "evlrg": 0, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "evadd": 2817, 2024-04-24T23:18:04.670 INFO:teuthology.orchestra.run.smithi150.stdout: "evex": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "evtrm": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "ev": 2817, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "evexg": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "evexd": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "segadd": 5, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "segex": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "segtrm": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "seg": 5, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "segmjr": 2, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "segexg": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "segexd": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "expos": 4194304, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "wrpos": 9481014, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "rdpos": 0, 2024-04-24T23:18:04.671 INFO:teuthology.orchestra.run.smithi150.stdout: "jlat": { 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 158, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.367349645, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.002324997 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "replayed": 0 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_mem": { 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "ino": 22, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "ino+": 1013, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "ino-": 991, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "dir": 21, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "dir+": 113, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "dir-": 92, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "dn": 18, 2024-04-24T23:18:04.672 INFO:teuthology.orchestra.run.smithi150.stdout: "dn+": 1997, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "dn-": 1979, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "cap": 1, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "cap+": 982, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "cap-": 981, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "rss": 63420, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "heap": 223612 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_server": { 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_client_request": 2697, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "dispatch_server_request": 18, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_request": 1588, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_client_session": 3, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "handle_peer_request": 693, 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "req_create_latency": { 2024-04-24T23:18:04.673 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getattr_latency": { 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getfilelock_latency": { 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.674 INFO:teuthology.orchestra.run.smithi150.stdout: "req_link_latency": { 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookup_latency": { 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 572, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.183244344, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000320357 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookuphash_latency": { 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.675 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupino_latency": { 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 1, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000115836, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000115836 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupname_latency": { 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupparent_latency": { 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.676 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lookupsnap_latency": { 2024-04-24T23:18:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "req_lssnap_latency": { 2024-04-24T23:18:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.677 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mkdir_latency": { 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mknod_latency": { 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "req_mksnap_latency": { 2024-04-24T23:18:04.678 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "req_open_latency": { 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "req_readdir_latency": { 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 51, 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.033743570, 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000661638 2024-04-24T23:18:04.679 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rename_latency": { 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "req_renamesnap_latency": { 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "req_snapdiff_latency": { 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.680 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmdir_latency": { 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 52, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.735651612, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.014147146 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmsnap_latency": { 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "req_rmxattr_latency": { 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.681 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setattr_latency": { 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setdirlayout_latency": { 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setfilelock_latency": { 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.682 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setlayout_latency": { 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "req_setxattr_latency": { 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "req_symlink_latency": { 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.683 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "req_unlink_latency": { 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 912, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 1.538929033, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.001687422 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "req_getvxattr_latency": { 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.684 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_sessions": { 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "session_count": 1, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "session_add": 1, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "session_remove": 0, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_open": 1, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "sessions_stale": 0, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "total_load": 1471, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "average_load": 1471, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "avg_session_uptime": 187, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "mdthresh_evicted": 0 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "mempool": { 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "bloom_filter_items": 0, 2024-04-24T23:18:04.685 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_extent_items": 0, 2024-04-24T23:18:04.686 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_blob_items": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_txc_items": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluestore_writing_items": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_bytes": 0, 2024-04-24T23:18:04.687 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_items": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_bytes": 291300, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_anon_items": 113, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "buffer_meta_items": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_bytes": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_items": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_mapbl_items": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_pglog_items": 0, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_bytes": 9176, 2024-04-24T23:18:04.688 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_items": 176, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_bytes": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "pgmap_items": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_bytes": 89733, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "mds_co_items": 381, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_bytes": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_1_items": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_bytes": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "unittest_2_items": 0 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "objecter": { 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "op_active": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "op_laggy": 0, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send": 2340, 2024-04-24T23:18:04.689 INFO:teuthology.orchestra.run.smithi150.stdout: "op_send_bytes": 5499764, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "op_resend": 33, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "op_reply": 2340, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "op_latency": { 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2340, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 7.216857244, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.003084127 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "op_inflight": 0, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "oplen_avg": { 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 2340, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 2388 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "op": 2340, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "op_r": 200, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "op_w": 2140, 2024-04-24T23:18:04.690 INFO:teuthology.orchestra.run.smithi150.stdout: "op_rmw": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "op_pg": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_stat": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_create": 10, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_read": 31, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_write": 189, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writefull": 21, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_writesame": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_append": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_zero": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_truncate": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_delete": 1889, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_mapext": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_sparse_read": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_clonerange": 0, 2024-04-24T23:18:04.691 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_getxattr": 169, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_setxattr": 20, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_rmxattr": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_call": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_watch": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_notify": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "osdop_other": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_active": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_send": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_resend": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "linger_ping": 0, 2024-04-24T23:18:04.692 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_active": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_send": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "poolop_resend": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_active": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_send": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "poolstat_resend": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_active": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_send": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "statfs_resend": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "command_active": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "command_send": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "command_resend": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "map_epoch": 94, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "map_full": 0, 2024-04-24T23:18:04.693 INFO:teuthology.orchestra.run.smithi150.stdout: "map_inc": 4, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_sessions": 12, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_open": 12, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_session_close": 0, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "osd_laggy": 0, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_wr": 46, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_rd": 0, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_del": 13 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "oft": { 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_objs": 1, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_updates": 0, 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: "omap_total_removes": 0 2024-04-24T23:18:04.694 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "osd-slow-ops": { 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "slow_ops_count": 0 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "purge_queue": { 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops": 0, 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing": 0, 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executing_high_water": 8, 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed_ops": 1876, 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_executed": 964, 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: "pq_item_in_journal": 0 2024-04-24T23:18:04.695 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 7027, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 3155359, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 7027, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 7027, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 3155359, 2024-04-24T23:18:04.696 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_bytes": { 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 104857600, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2340, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 5604424, 2024-04-24T23:18:04.697 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2340, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 5604424, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-objecter_ops": { 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 1024, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 0, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 0, 2024-04-24T23:18:04.698 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 2340, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 2340, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 2340, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 2340, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-24T23:18:04.699 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 964, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 104660, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 964, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 31, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 104660, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: }, 2024-04-24T23:18:04.700 INFO:teuthology.orchestra.run.smithi150.stdout: "throttle-write_buf_throttle-0x55f22d6f0500": { 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "val": 0, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "max": 3758096384, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "get_started": 0, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "get": 2817, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "get_sum": 5286710, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "get_or_fail_success": 2817, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "take": 0, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "take_sum": 0, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "put": 158, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "put_sum": 5286710, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "wait": { 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "avgcount": 0, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "sum": 0.000000000, 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: "avgtime": 0.000000000 2024-04-24T23:18:04.701 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.702 INFO:teuthology.orchestra.run.smithi150.stdout: } 2024-04-24T23:18:04.702 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-04-24T23:18:05.449 INFO:tasks.check_counter:Daemon mds.k mds.exported=18 2024-04-24T23:18:05.450 INFO:tasks.check_counter:Daemon mds.k mds.imported=18 2024-04-24T23:18:05.450 DEBUG:tasks.check_counter:Getting stats from c 2024-04-24T23:18:05.450 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-24T23:18:05.700 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:18:06.382 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:06 smithi187 ceph-mon[33292]: pgmap v463: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:06.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:06 smithi112 ceph-mon[29186]: pgmap v463: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:06.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:06 smithi150 ceph-mon[32247]: pgmap v463: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:07.990 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-24T23:18:07.990 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:18:07.990 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 681, 2024-04-24T23:18:07.990 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 681, 2024-04-24T23:18:07.990 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 24840238, 2024-04-24T23:18:07.990 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 215744, 2024-04-24T23:18:07.990 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 7, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 3, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.080491937, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.020462710, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.032832586, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.020288662, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 680, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.028390938, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000041751 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:07.991 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 448 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 118, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 193, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 25202741, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 239304, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 6, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.024944633, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.006018076, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.014801043, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.002657516, 2024-04-24T23:18:07.992 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 191, 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.011447408, 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000059934 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 202032 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:18:07.993 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 448, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 429, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 21615192, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 163669, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 5, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.058779109, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.016303001, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.029724428, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.012486614, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 428, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.018070449, 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000042220 2024-04-24T23:18:07.994 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 563872, 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 62544 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.995 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1795, 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.048433926, 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000026982 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:07.996 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-04-24T23:18:07.997 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 5604, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 1, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 5603, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 5602, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 2, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2024-04-24T23:18:07.998 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-04-24T23:18:07.999 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 4, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 234, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:18:08.000 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-04-24T23:18:08.001 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:18:08.002 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2024-04-24T23:18:08.003 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 34209, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 42, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 4, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-04-24T23:18:08.004 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 4194304, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 74964871, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 74962883, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 34209 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 32, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 10724, 2024-04-24T23:18:08.005 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 10692, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 38, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 759, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 721, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 5604, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 13365, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 7761, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 66488, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 215420 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-04-24T23:18:08.006 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.007 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.008 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.009 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.011 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.011 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-04-24T23:18:08.011 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.011 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.011 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.011 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.013 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-04-24T23:18:08.013 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.013 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.013 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.013 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.013 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.015 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.016 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-04-24T23:18:08.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.018 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.019 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 3, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 2, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2024-04-24T23:18:08.020 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 1318, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:18:08.021 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:18:08.022 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:18:08.023 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 221909, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 74, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 9176, 2024-04-24T23:18:08.024 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 176, 2024-04-24T23:18:08.025 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:18:08.025 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:18:08.025 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-04-24T23:18:08.025 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-04-24T23:18:08.025 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 4069554, 2024-04-24T23:18:08.025 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 25324, 2024-04-24T23:18:08.025 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 1112, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 26, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 4, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 1112, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1112, 2024-04-24T23:18:08.026 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.805987128, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000724808 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1112, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1113 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 1112, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 1112, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 694, 2024-04-24T23:18:08.027 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 415, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 2, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:18:08.028 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-04-24T23:18:08.029 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 94, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 4, 2024-04-24T23:18:08.030 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 9, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 9, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:18:08.031 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 0, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-04-24T23:18:08.032 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 1243, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 71536747, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 1243, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 1243, 2024-04-24T23:18:08.033 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 71536747, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:08.034 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 1112, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 70781355, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 1112, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 70781355, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:08.035 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 1112, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 1112, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 1112, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 1112, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:18:08.036 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.037 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55b503ea86e0": { 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-04-24T23:18:08.038 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:08.039 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:08.039 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:08.039 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:08.039 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:08.039 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:08.039 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-24T23:18:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:08 smithi187 ceph-mon[33292]: pgmap v464: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:08.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:08 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:08.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:08 smithi112 ceph-mon[29186]: pgmap v464: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:08.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:08 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:08.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:08 smithi150 ceph-mon[32247]: pgmap v464: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:08.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:08 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:08.785 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-24T23:18:08.785 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-24T23:18:08.785 DEBUG:tasks.check_counter:Getting stats from f 2024-04-24T23:18:08.786 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-24T23:18:09.038 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:18:09.344 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:09 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:09.344 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:09 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:09.344 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:09 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:18:09.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:09 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:09.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:09 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:09.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:09 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:18:09.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:09 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:09.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:09 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:09.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:09 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' 2024-04-24T23:18:10.233 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:10 smithi187 ceph-mon[33292]: pgmap v465: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:10.469 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:10 smithi150 ceph-mon[32247]: pgmap v465: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:10.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:10 smithi112 ceph-mon[29186]: pgmap v465: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:10.804 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-24T23:18:10.804 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:18:10.804 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 449, 2024-04-24T23:18:10.804 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 428, 2024-04-24T23:18:10.804 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 672816, 2024-04-24T23:18:10.804 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 163642, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 5, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 1, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.054315718, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.016551067, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.021641759, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.014528269, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 428, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.022154270, 2024-04-24T23:18:10.805 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000051762 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 566976, 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 63472 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:18:10.806 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 23, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 99, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 4200084, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 210882, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 5, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.010457778, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.003475997, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.005501452, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.000518365, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 97, 2024-04-24T23:18:10.807 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.005832285, 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000060126 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 203824 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.808 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 598, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 600, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 16471894, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 190171, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 5, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.049645564, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.014638463, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.017649426, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.014188848, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 598, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.022475519, 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000037584 2024-04-24T23:18:10.809 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-04-24T23:18:10.810 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1593, 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.042853934, 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000026901 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.811 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-04-24T23:18:10.812 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 2892, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 2892, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 2892, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 1, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2024-04-24T23:18:10.813 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-04-24T23:18:10.814 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:18:10.815 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-04-24T23:18:10.816 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-04-24T23:18:10.817 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 9812, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2024-04-24T23:18:10.818 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 13, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 2, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 4194304, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 24702457, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 24701858, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 9812 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.819 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 11, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 2923, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 2912, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 9, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 189, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 180, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 2892, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 4744, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 1852, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 60040, 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 215420 2024-04-24T23:18:10.820 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.821 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-04-24T23:18:10.822 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.823 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.824 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-04-24T23:18:10.825 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.825 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.825 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.825 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.825 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-04-24T23:18:10.825 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.825 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-04-24T23:18:10.826 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.827 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.828 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.829 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.830 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.831 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.832 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 2, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 1, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 1320, 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-04-24T23:18:10.833 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:18:10.834 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-04-24T23:18:10.835 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 198137, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 56, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:18:10.836 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 9176, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 176, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-04-24T23:18:10.837 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 2072376, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 11674, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 935, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 21, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 3, 2024-04-24T23:18:10.838 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 935, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 935, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.364978208, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001459869 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 935, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 936 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 935, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 935, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-04-24T23:18:10.839 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 578, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 355, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 1, 2024-04-24T23:18:10.840 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-04-24T23:18:10.841 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 94, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-04-24T23:18:10.842 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 4, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 6, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 6, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2024-04-24T23:18:10.843 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 0, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:10.844 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 1066, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 21239294, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 1066, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 1066, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 21239294, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.845 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 935, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 20513106, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 935, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 20513106, 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:10.846 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 935, 2024-04-24T23:18:10.847 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 935, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 935, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 935, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:10.848 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55649ad426e0": { 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:10.849 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:10.850 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-24T23:18:11.612 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-24T23:18:11.612 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-24T23:18:11.612 DEBUG:tasks.check_counter:Getting stats from i 2024-04-24T23:18:11.613 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-24T23:18:11.849 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:18:12.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:12 smithi187 ceph-mon[33292]: pgmap v466: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:12.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:12 smithi112 ceph-mon[29186]: pgmap v466: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:12.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:12 smithi150 ceph-mon[32247]: pgmap v466: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:13.793 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 9971, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 19281, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 3621566, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 24954905, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 23, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.715207394, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.331828282, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.395583650, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.072650781, 2024-04-24T23:18:13.794 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 19275, 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.740297077, 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000038407 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 20927, 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.003302519, 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000157 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 580672, 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 71232 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:18:13.795 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 3278, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 3493, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 2918662, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 1450669, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 10, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.234211032, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.082047002, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.101034421, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.051611984, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3497, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.217291797, 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000062136 2024-04-24T23:18:13.796 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2708, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000500908, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000184 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 205328 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 2876, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 2877, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 1122657, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 5345281, 2024-04-24T23:18:13.797 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 12, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 7, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.208647139, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.067781771, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.056486926, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.067721099, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2875, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.205072138, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000071329 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 84, 2024-04-24T23:18:13.798 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000015431, 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000183 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1091, 2024-04-24T23:18:13.799 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 5.620459411, 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.005151658 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 5167, 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.109290595, 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000214687 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 4354, 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 4351, 2024-04-24T23:18:13.800 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4351, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 8.040994752, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001848079 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 3, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 20, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 1, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 16, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 0, 2024-04-24T23:18:13.801 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 2, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 14, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 14, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 86, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 1, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 1, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 3, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 4973, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 4618, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 6, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2024-04-24T23:18:13.802 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 2000, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 2, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 12, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 62, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 5, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 1824, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 1245, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 2794, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 107894170, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 1245, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:18:13.803 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 57, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 57, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 1188, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 236, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 4203, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 2788, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 1945, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 37, 2024-04-24T23:18:13.804 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 3329 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 6, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 2884, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 2884, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-04-24T23:18:13.805 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 12, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 1, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2024-04-24T23:18:13.806 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 9812, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 9812, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 13, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 13, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 2, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-04-24T23:18:13.807 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 4194304, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 24702457, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 0, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 173, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 2.565763027, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.014831000 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 0 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 20, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 2918, 2024-04-24T23:18:13.808 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 2898, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 19, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 185, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 166, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 16, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 4748, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 4732, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 1, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 2890, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 2889, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 79108, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 223612 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 5038, 2024-04-24T23:18:13.809 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 4, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 4354, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 7, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 359, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1060, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.467111070, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001384067 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.001087540, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000543770 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.810 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 262, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.085563123, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000326576 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.811 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.812 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-04-24T23:18:13.813 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.814 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.814 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.814 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.814 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-04-24T23:18:13.814 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.814 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.814 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.814 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.815 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 143, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.091188294, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000637680 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.816 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 96, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.959440459, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.020410838 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.817 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-04-24T23:18:13.818 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.819 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2788, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 4.436604266, 2024-04-24T23:18:13.820 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001591321 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 2, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 1, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 3056, 2024-04-24T23:18:13.821 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 3056, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 205, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:18:13.822 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:18:13.823 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 298927, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 124, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 616, 2024-04-24T23:18:13.824 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 7, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 9176, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 176, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 81621, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 385, 2024-04-24T23:18:13.825 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 7313, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 21449976, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 33, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 7313, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 7313, 2024-04-24T23:18:13.826 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 25.196597823, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003445453 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 7313, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 7388 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 7313, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 1283, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 6030, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 0, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 10, 2024-04-24T23:18:13.827 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 38, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 211, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 25, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 1, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 5728, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 1245, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 20, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-04-24T23:18:13.828 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-04-24T23:18:13.829 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 94, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 4, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 12, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2024-04-24T23:18:13.830 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 89, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 0, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 21 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 1, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.831 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 17, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 6, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 5712, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 2884, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 16121, 2024-04-24T23:18:13.832 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 6200493, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 16121, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 16121, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 6200493, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:13.833 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 7313, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 21763604, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 7313, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 21763604, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.834 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 7313, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 7313, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 7313, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 7313, 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:13.835 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 2884, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 313628, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 2884, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:13.836 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 38, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 313628, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55be51c6c500": { 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 9812, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 20508153, 2024-04-24T23:18:13.837 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 9812, 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 173, 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 20508153, 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:13.838 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-24T23:18:14.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:14 smithi187 ceph-mon[33292]: pgmap v467: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:18:14.526 INFO:tasks.check_counter:Daemon mds.i mds.exported=12 2024-04-24T23:18:14.526 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-04-24T23:18:14.526 DEBUG:tasks.check_counter:Getting stats from l 2024-04-24T23:18:14.526 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid e62ba378-028e-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-24T23:18:14.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:14 smithi112 ceph-mon[29186]: pgmap v467: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:18:14.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:14 smithi150 ceph-mon[32247]: pgmap v467: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-24T23:18:14.762 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/mon.c/config 2024-04-24T23:18:16.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:16 smithi187 ceph-mon[33292]: pgmap v468: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T23:18:16.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:16 smithi112 ceph-mon[29186]: pgmap v468: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 3142, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 6962, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 1135862, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 7961430, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 23, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.237737800, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.097450176, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.113527851, 2024-04-24T23:18:16.545 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.025007688, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6958, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.283124159, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000040690 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6960, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.001207117, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000173 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.546 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 3536, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 4574, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 2835611, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 1363263, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 9, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.254936851, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.094151530, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.138132679, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.037029786, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4576, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.186217764, 2024-04-24T23:18:16.547 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000040694 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 5324, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000859225, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000161 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 559456, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 74288 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 926, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1033, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 317149, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 2865088, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 12, 2024-04-24T23:18:16.548 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.093696732, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.041246886, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.023685005, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.023070046, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1031, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.082557782, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000080075 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 237, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000036632, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000154 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-24T23:18:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 206208 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 457, 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.161322102, 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002541186 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.550 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2280, 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.302243708, 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000132563 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 2010, 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 2009, 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2009, 2024-04-24T23:18:16.551 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 3.140384028, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001563157 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 1, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 11, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 22, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 6, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 16, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 0, 2024-04-24T23:18:16.552 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 13, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 49, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 3, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 3493, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 2731, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 15, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 2000, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 2, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 15, 2024-04-24T23:18:16.553 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 16, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 1227, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 123, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 2794, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 107894170, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 123, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-04-24T23:18:16.554 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 123, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 2, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 150, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 1368, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 266, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 123, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 13, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 2009 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2024-04-24T23:18:16.555 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 17, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 1227, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 1227, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-04-24T23:18:16.556 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 15, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 3467, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 3467, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-04-24T23:18:16.557 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 6, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 6, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 2, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 4194304, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 10533821, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 0, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 155, 2024-04-24T23:18:16.558 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.368384452, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002376673 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 0 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 26, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 1278, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 1252, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 25, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 123, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 98, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 22, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 2525, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 2503, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2024-04-24T23:18:16.559 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 1242, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 1242, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 64704, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 223612 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 2974, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 22, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 2010, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 2, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 781, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.560 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.561 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.561 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-04-24T23:18:16.561 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1, 2024-04-24T23:18:16.561 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.001098253, 2024-04-24T23:18:16.561 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001098253 2024-04-24T23:18:16.561 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.561 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-04-24T23:18:16.561 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 704, 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.227198774, 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000322725 2024-04-24T23:18:16.562 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.563 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.564 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.565 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 77, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.060411063, 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000784559 2024-04-24T23:18:16.566 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.567 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 63, 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.953474120, 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.015134509 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-04-24T23:18:16.568 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.569 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.569 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.569 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.569 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-04-24T23:18:16.569 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.569 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.569 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.569 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.570 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.571 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1164, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.898201818, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001630757 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2024-04-24T23:18:16.572 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 1, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 0, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 1537, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 1537, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 1326, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-04-24T23:18:16.573 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-04-24T23:18:16.574 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-04-24T23:18:16.575 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 307773, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 118, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-04-24T23:18:16.576 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 9176, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 176, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 106157, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 422, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-04-24T23:18:16.577 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 2816, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 6561500, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 33, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 2816, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2816, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 9.066855207, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003219763 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2816, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 2871 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.578 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 2816, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 157, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 2659, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 0, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 10, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 34, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 189, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 21, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-04-24T23:18:16.579 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 2404, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 123, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 20, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-04-24T23:18:16.580 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-04-24T23:18:16.581 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 94, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 4, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 12, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 54, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 0, 2024-04-24T23:18:16.582 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 16 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 1, 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 10, 2024-04-24T23:18:16.583 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 6, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 2391, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 1227, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 7600, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 3593419, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 7600, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:16.584 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 7600, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 3593419, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:16.585 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 2816, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 6694739, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 2816, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 6694739, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-04-24T23:18:16.586 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 2816, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 2816, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 2816, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 2816, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.587 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 1227, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 133239, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 1227, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 34, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 133239, 2024-04-24T23:18:16.588 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x562240914500": { 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 3467, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 6339517, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 3467, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 155, 2024-04-24T23:18:16.589 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 6339517, 2024-04-24T23:18:16.590 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-24T23:18:16.590 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-24T23:18:16.590 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-24T23:18:16.590 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-24T23:18:16.590 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.590 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-24T23:18:16.590 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-24T23:18:16.590 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:16 smithi150 ceph-mon[32247]: pgmap v468: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 10 op/s 2024-04-24T23:18:17.352 INFO:tasks.check_counter:Daemon mds.l mds.exported=15 2024-04-24T23:18:17.352 INFO:tasks.check_counter:Daemon mds.l mds.imported=16 2024-04-24T23:18:17.352 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-24T23:18:17.362 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-24T23:18:17.362 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:18:17.362 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:18:17.382 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-24T23:18:17.382 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:18:17.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:18:17.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.593+0000 7f960b1a5640 1 -- 172.21.15.112:0/3392759082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96041006a0 msgr2=0x7f9604100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:17.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.593+0000 7f960b1a5640 1 --2- 172.21.15.112:0/3392759082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96041006a0 0x7f9604100ac0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9600009900 tx=0x7f960002a410 comp rx=0 tx=0).stop 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.593+0000 7f960b1a5640 1 -- 172.21.15.112:0/3392759082 shutdown_connections 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.593+0000 7f960b1a5640 1 --2- 172.21.15.112:0/3392759082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9604102250 0x7f9604108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.593+0000 7f960b1a5640 1 --2- 172.21.15.112:0/3392759082 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604101890 0x7f9604101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.593+0000 7f960b1a5640 1 --2- 172.21.15.112:0/3392759082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96041006a0 0x7f9604100ac0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.593+0000 7f960b1a5640 1 -- 172.21.15.112:0/3392759082 >> 172.21.15.112:0/3392759082 conn(0x7f96040fbc30 msgr2=0x7f96040fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 -- 172.21.15.112:0/3392759082 shutdown_connections 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 -- 172.21.15.112:0/3392759082 wait complete. 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 Processor -- start 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 -- start start 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96041006a0 0x7f9604196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604101890 0x7f9604196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604102250 0x7f960419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:17.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f960410c450 con 0x7f9604102250 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f960410c2d0 con 0x7f9604101890 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f960b1a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f960410c5d0 con 0x7f96041006a0 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f9609742640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604102250 0x7f960419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f9609742640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604102250 0x7f960419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52000/0 (socket says 172.21.15.112:52000) 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.594+0000 7f9609742640 1 -- 172.21.15.112:0/467016053 learned_addr learned my addr 172.21.15.112:0/467016053 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f95fbfff640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604101890 0x7f9604196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f9608f41640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96041006a0 0x7f9604196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f9609742640 1 -- 172.21.15.112:0/467016053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96041006a0 msgr2=0x7f9604196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f9609742640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96041006a0 0x7f9604196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f9609742640 1 -- 172.21.15.112:0/467016053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604101890 msgr2=0x7f9604196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f9609742640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604101890 0x7f9604196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f9609742640 1 -- 172.21.15.112:0/467016053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960419e650 con 0x7f9604102250 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f9608f41640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96041006a0 0x7f9604196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f9609742640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604102250 0x7f960419dfe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00eb10 tx=0x7f95fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:17.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f95f9ffb640 1 -- 172.21.15.112:0/467016053 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc004b20 con 0x7f9604102250 2024-04-24T23:18:17.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960419e940 con 0x7f9604102250 2024-04-24T23:18:17.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f95f9ffb640 1 -- 172.21.15.112:0/467016053 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f95fc004cc0 con 0x7f9604102250 2024-04-24T23:18:17.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f95f9ffb640 1 -- 172.21.15.112:0/467016053 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0107f0 con 0x7f9604102250 2024-04-24T23:18:17.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.595+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960419eee0 con 0x7f9604102250 2024-04-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.596+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96041078f0 con 0x7f9604102250 2024-04-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.597+0000 7f95f9ffb640 1 -- 172.21.15.112:0/467016053 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f95fc005020 con 0x7f9604102250 2024-04-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.598+0000 7f95f9ffb640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95ec0a2b90 0x7f95ec0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.598+0000 7f9608f41640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95ec0a2b90 0x7f95ec0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.598+0000 7f95f9ffb640 1 -- 172.21.15.112:0/467016053 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f95fc0cfae0 con 0x7f9604102250 2024-04-24T23:18:17.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.598+0000 7f9608f41640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95ec0a2b90 0x7f95ec0a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9600000c00 tx=0x7f9600009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:17.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.602+0000 7f95f9ffb640 1 -- 172.21.15.112:0/467016053 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f95fc09c650 con 0x7f9604102250 2024-04-24T23:18:17.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.778+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9604062340 con 0x7f9604102250 2024-04-24T23:18:17.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.780+0000 7f95f9ffb640 1 -- 172.21.15.112:0/467016053 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7f95fc09bf50 con 0x7f9604102250 2024-04-24T23:18:17.782 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:18:17.782 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:18:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95ec0a2b90 msgr2=0x7f95ec0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95ec0a2b90 0x7f95ec0a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9600000c00 tx=0x7f9600009560 comp rx=0 tx=0).stop 2024-04-24T23:18:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604102250 msgr2=0x7f960419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604102250 0x7f960419dfe0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00eb10 tx=0x7f95fc00c2d0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 shutdown_connections 2024-04-24T23:18:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f95ec0a2b90 0x7f95ec0a5050 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9604102250 0x7f960419dfe0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9604101890 0x7f9604196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 --2- 172.21.15.112:0/467016053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96041006a0 0x7f9604196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 >> 172.21.15.112:0/467016053 conn(0x7f96040fbc30 msgr2=0x7f9604104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 shutdown_connections 2024-04-24T23:18:17.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.782+0000 7f960b1a5640 1 -- 172.21.15.112:0/467016053 wait complete. 2024-04-24T23:18:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:07:51.018846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-24T23:12:25.486401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-24T23:12:26.512604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-24T23:12:30.239863+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:18:17.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:18:17.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.895+0000 7f4656bd5640 1 -- 172.21.15.112:0/2518102839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650101870 msgr2=0x7f4650101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:17.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.895+0000 7f4656bd5640 1 --2- 172.21.15.112:0/2518102839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650101870 0x7f4650101cf0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f463c009900 tx=0x7f463c02a410 comp rx=0 tx=0).stop 2024-04-24T23:18:17.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.895+0000 7f4656bd5640 1 -- 172.21.15.112:0/2518102839 shutdown_connections 2024-04-24T23:18:17.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.895+0000 7f4656bd5640 1 --2- 172.21.15.112:0/2518102839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4650102230 0x7f4650108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.895+0000 7f4656bd5640 1 --2- 172.21.15.112:0/2518102839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650101870 0x7f4650101cf0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 --2- 172.21.15.112:0/2518102839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650100680 0x7f4650100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 -- 172.21.15.112:0/2518102839 >> 172.21.15.112:0/2518102839 conn(0x7f46500fbbf0 msgr2=0x7f46500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 -- 172.21.15.112:0/2518102839 shutdown_connections 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 -- 172.21.15.112:0/2518102839 wait complete. 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 Processor -- start 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 -- start start 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650100680 0x7f4650196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4650101870 0x7f4650196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650102230 0x7f465019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f465010c430 con 0x7f4650102230 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f465010c2b0 con 0x7f4650100680 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4656bd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f465010c5b0 con 0x7f4650101870 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4655172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650102230 0x7f465019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4655172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650102230 0x7f465019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52020/0 (socket says 172.21.15.112:52020) 2024-04-24T23:18:17.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4655172640 1 -- 172.21.15.112:0/916480874 learned_addr learned my addr 172.21.15.112:0/916480874 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.896+0000 7f4654971640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650100680 0x7f4650196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f464ffff640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4650101870 0x7f4650196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f4655172640 1 -- 172.21.15.112:0/916480874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4650101870 msgr2=0x7f4650196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f4655172640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4650101870 0x7f4650196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f4655172640 1 -- 172.21.15.112:0/916480874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650100680 msgr2=0x7f4650196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f4655172640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650100680 0x7f4650196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f4655172640 1 -- 172.21.15.112:0/916480874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465019e690 con 0x7f4650102230 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f464ffff640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4650101870 0x7f4650196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f4655172640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650102230 0x7f465019e020 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f464800ebb0 tx=0x7f4648007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f464dffb640 1 -- 172.21.15.112:0/916480874 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648007ca0 con 0x7f4650102230 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f464dffb640 1 -- 172.21.15.112:0/916480874 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4648007e40 con 0x7f4650102230 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46500688d0 con 0x7f4650102230 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f464dffb640 1 -- 172.21.15.112:0/916480874 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648005940 con 0x7f4650102230 2024-04-24T23:18:17.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.897+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4650068e20 con 0x7f4650102230 2024-04-24T23:18:17.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.898+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4650068560 con 0x7f4650102230 2024-04-24T23:18:17.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.899+0000 7f464dffb640 1 -- 172.21.15.112:0/916480874 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4648005020 con 0x7f4650102230 2024-04-24T23:18:17.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.899+0000 7f464dffb640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f46400a2b90 0x7f46400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:17.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.900+0000 7f4654971640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f46400a2b90 0x7f46400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:17.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.900+0000 7f464dffb640 1 -- 172.21.15.112:0/916480874 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f46480d0950 con 0x7f4650102230 2024-04-24T23:18:17.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.900+0000 7f4654971640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f46400a2b90 0x7f46400a5050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4644005fd0 tx=0x7f464400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:17.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:17.904+0000 7f464dffb640 1 -- 172.21.15.112:0/916480874 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f464809d4c0 con 0x7f4650102230 2024-04-24T23:18:18.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.078+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f465019e8c0 con 0x7f4650102230 2024-04-24T23:18:18.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.079+0000 7f464dffb640 1 -- 172.21.15.112:0/916480874 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f464809cdc0 con 0x7f4650102230 2024-04-24T23:18:18.081 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-24T23:18:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f46400a2b90 msgr2=0x7f46400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f46400a2b90 0x7f46400a5050 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4644005fd0 tx=0x7f464400b410 comp rx=0 tx=0).stop 2024-04-24T23:18:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650102230 msgr2=0x7f465019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650102230 0x7f465019e020 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f464800ebb0 tx=0x7f4648007590 comp rx=0 tx=0).stop 2024-04-24T23:18:18.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 shutdown_connections 2024-04-24T23:18:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f46400a2b90 0x7f46400a5050 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4650102230 0x7f465019e020 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4650101870 0x7f4650196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 --2- 172.21.15.112:0/916480874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4650100680 0x7f4650196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 >> 172.21.15.112:0/916480874 conn(0x7f46500fbbf0 msgr2=0x7f4650104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 shutdown_connections 2024-04-24T23:18:18.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.081+0000 7f4656bd5640 1 -- 172.21.15.112:0/916480874 wait complete. 2024-04-24T23:18:18.091 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:18 smithi112 ceph-mon[29186]: pgmap v469: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:18.091 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:18 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/467016053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:18:18.091 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:18 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/916480874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:18:18.094 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-24T23:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.193+0000 7f14c4529640 1 -- 172.21.15.112:0/1864896057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc107500 msgr2=0x7f14bc1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.193+0000 7f14c4529640 1 --2- 172.21.15.112:0/1864896057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc107500 0x7f14bc1098f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009900 tx=0x7f14b402a410 comp rx=0 tx=0).stop 2024-04-24T23:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.193+0000 7f14c4529640 1 -- 172.21.15.112:0/1864896057 shutdown_connections 2024-04-24T23:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.193+0000 7f14c4529640 1 --2- 172.21.15.112:0/1864896057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc107500 0x7f14bc1098f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.193+0000 7f14c4529640 1 --2- 172.21.15.112:0/1864896057 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14bc104bd0 0x7f14bc106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.193+0000 7f14c4529640 1 --2- 172.21.15.112:0/1864896057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14bc1022a0 0x7f14bc104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.193+0000 7f14c4529640 1 -- 172.21.15.112:0/1864896057 >> 172.21.15.112:0/1864896057 conn(0x7f14bc0fbbf0 msgr2=0x7f14bc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 -- 172.21.15.112:0/1864896057 shutdown_connections 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 -- 172.21.15.112:0/1864896057 wait complete. 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 Processor -- start 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 -- start start 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc1022a0 0x7f14bc197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14bc104bd0 0x7f14bc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14bc107500 0x7f14bc19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc10d550 con 0x7f14bc1022a0 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc10d3d0 con 0x7f14bc107500 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c4529640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc10d6d0 con 0x7f14bc104bd0 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c22c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc1022a0 0x7f14bc197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c22c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc1022a0 0x7f14bc197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52048/0 (socket says 172.21.15.112:52048) 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c22c5640 1 -- 172.21.15.112:0/3742883796 learned_addr learned my addr 172.21.15.112:0/3742883796 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.194+0000 7f14c2ac6640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14bc107500 0x7f14bc19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c22c5640 1 -- 172.21.15.112:0/3742883796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14bc104bd0 msgr2=0x7f14bc1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c22c5640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14bc104bd0 0x7f14bc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c22c5640 1 -- 172.21.15.112:0/3742883796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14bc107500 msgr2=0x7f14bc19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c22c5640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14bc107500 0x7f14bc19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c22c5640 1 -- 172.21.15.112:0/3742883796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc19f7b0 con 0x7f14bc1022a0 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c2ac6640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14bc107500 0x7f14bc19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c22c5640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc1022a0 0x7f14bc197b90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f14b800ea80 tx=0x7f14b800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14b37fe640 1 -- 172.21.15.112:0/3742883796 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b800ce40 con 0x7f14bc1022a0 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc19faa0 con 0x7f14bc1022a0 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14b37fe640 1 -- 172.21.15.112:0/3742883796 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f14b8004b20 con 0x7f14bc1022a0 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14b37fe640 1 -- 172.21.15.112:0/3742883796 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8011830 con 0x7f14bc1022a0 2024-04-24T23:18:18.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.195+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc19fff0 con 0x7f14bc1022a0 2024-04-24T23:18:18.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.196+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc0fd6e0 con 0x7f14bc1022a0 2024-04-24T23:18:18.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.197+0000 7f14b37fe640 1 -- 172.21.15.112:0/3742883796 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f14b8004cd0 con 0x7f14bc1022a0 2024-04-24T23:18:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.198+0000 7f14b37fe640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14a80a2b90 0x7f14a80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.198+0000 7f14c1ac4640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14a80a2b90 0x7f14a80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.198+0000 7f14b37fe640 1 -- 172.21.15.112:0/3742883796 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f14b80cfb20 con 0x7f14bc1022a0 2024-04-24T23:18:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.198+0000 7f14c1ac4640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14a80a2b90 0x7f14a80a5050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0058c0 tx=0x7f14ac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:18.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.201+0000 7f14b37fe640 1 -- 172.21.15.112:0/3742883796 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14b80cfe20 con 0x7f14bc1022a0 2024-04-24T23:18:18.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.375+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f14bc02d400 con 0x7f14bc1022a0 2024-04-24T23:18:18.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.376+0000 7f14b37fe640 1 -- 172.21.15.112:0/3742883796 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f14b8016030 con 0x7f14bc1022a0 2024-04-24T23:18:18.378 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14a80a2b90 msgr2=0x7f14a80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14a80a2b90 0x7f14a80a5050 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0058c0 tx=0x7f14ac005850 comp rx=0 tx=0).stop 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc1022a0 msgr2=0x7f14bc197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc1022a0 0x7f14bc197b90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f14b800ea80 tx=0x7f14b800ef40 comp rx=0 tx=0).stop 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 shutdown_connections 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f14a80a2b90 0x7f14a80a5050 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f14bc107500 0x7f14bc19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14bc104bd0 0x7f14bc1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 --2- 172.21.15.112:0/3742883796 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14bc1022a0 0x7f14bc197b90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 >> 172.21.15.112:0/3742883796 conn(0x7f14bc0fbbf0 msgr2=0x7f14bc102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 shutdown_connections 2024-04-24T23:18:18.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.378+0000 7f14c4529640 1 -- 172.21.15.112:0/3742883796 wait complete. 2024-04-24T23:18:18.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-24T23:18:18.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:18 smithi187 ceph-mon[33292]: pgmap v469: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:18.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:18 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/467016053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:18:18.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:18 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/916480874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.489+0000 7f928b42a640 1 -- 172.21.15.112:0/4158825775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284102250 msgr2=0x7f9284108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 --2- 172.21.15.112:0/4158825775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284102250 0x7f9284108ae0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f927c009ca0 tx=0x7f927c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 -- 172.21.15.112:0/4158825775 shutdown_connections 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 --2- 172.21.15.112:0/4158825775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284102250 0x7f9284108ae0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 --2- 172.21.15.112:0/4158825775 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9284101890 0x7f9284101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 --2- 172.21.15.112:0/4158825775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f92841006a0 0x7f9284100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 -- 172.21.15.112:0/4158825775 >> 172.21.15.112:0/4158825775 conn(0x7f92840fbc30 msgr2=0x7f92840fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 -- 172.21.15.112:0/4158825775 shutdown_connections 2024-04-24T23:18:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 -- 172.21.15.112:0/4158825775 wait complete. 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 Processor -- start 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 -- start start 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92841006a0 0x7f9284197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284101890 0x7f9284198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9284102250 0x7f928419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f928410c450 con 0x7f9284101890 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f928410c2d0 con 0x7f92841006a0 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f928b42a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f928410c5d0 con 0x7f9284102250 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.490+0000 7f92889c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284101890 0x7f9284198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92889c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284101890 0x7f9284198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52068/0 (socket says 172.21.15.112:52068) 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92889c5640 1 -- 172.21.15.112:0/1394710056 learned_addr learned my addr 172.21.15.112:0/1394710056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92899c7640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9284102250 0x7f928419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92891c6640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92841006a0 0x7f9284197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92889c5640 1 -- 172.21.15.112:0/1394710056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9284102250 msgr2=0x7f928419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92889c5640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9284102250 0x7f928419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92889c5640 1 -- 172.21.15.112:0/1394710056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92841006a0 msgr2=0x7f9284197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92889c5640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92841006a0 0x7f9284197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92889c5640 1 -- 172.21.15.112:0/1394710056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928419f770 con 0x7f9284101890 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92889c5640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284101890 0x7f9284198090 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f928000d950 tx=0x7f928000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92767fc640 1 -- 172.21.15.112:0/1394710056 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280017070 con 0x7f9284101890 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92767fc640 1 -- 172.21.15.112:0/1394710056 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9280009c30 con 0x7f9284101890 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928419fa60 con 0x7f9284101890 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f92767fc640 1 -- 172.21.15.112:0/1394710056 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928000f740 con 0x7f9284101890 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.491+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928419ffb0 con 0x7f9284101890 2024-04-24T23:18:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.492+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92841a01e0 con 0x7f9284101890 2024-04-24T23:18:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.493+0000 7f92767fc640 1 -- 172.21.15.112:0/1394710056 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9280010a80 con 0x7f9284101890 2024-04-24T23:18:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.494+0000 7f92767fc640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f92700a2b90 0x7f92700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.494+0000 7f92891c6640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f92700a2b90 0x7f92700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.494+0000 7f92767fc640 1 -- 172.21.15.112:0/1394710056 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f92800cf080 con 0x7f9284101890 2024-04-24T23:18:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.494+0000 7f92891c6640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f92700a2b90 0x7f92700a5050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9278005fd0 tx=0x7f9278005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:18.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.499+0000 7f92767fc640 1 -- 172.21.15.112:0/1394710056 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9280009de0 con 0x7f9284101890 2024-04-24T23:18:18.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:18 smithi150 ceph-mon[32247]: pgmap v469: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-24T23:18:18.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:18 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/467016053' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:18:18.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:18 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/916480874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:18:18.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.673+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f92841a0450 con 0x7f9284101890 2024-04-24T23:18:18.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.674+0000 7f92767fc640 1 -- 172.21.15.112:0/1394710056 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f928009bbf0 con 0x7f9284101890 2024-04-24T23:18:18.676 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-24T23:18:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f92700a2b90 msgr2=0x7f92700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f92700a2b90 0x7f92700a5050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9278005fd0 tx=0x7f9278005850 comp rx=0 tx=0).stop 2024-04-24T23:18:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284101890 msgr2=0x7f9284198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284101890 0x7f9284198090 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f928000d950 tx=0x7f928000de10 comp rx=0 tx=0).stop 2024-04-24T23:18:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 shutdown_connections 2024-04-24T23:18:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f92700a2b90 0x7f92700a5050 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9284102250 0x7f928419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9284101890 0x7f9284198090 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 --2- 172.21.15.112:0/1394710056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92841006a0 0x7f9284197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 >> 172.21.15.112:0/1394710056 conn(0x7f92840fbc30 msgr2=0x7f92840fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 shutdown_connections 2024-04-24T23:18:18.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.676+0000 7f928b42a640 1 -- 172.21.15.112:0/1394710056 wait complete. 2024-04-24T23:18:18.689 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-24T23:18:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.787+0000 7fa417b1d640 1 -- 172.21.15.112:0/3500868097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 msgr2=0x7fa410108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 --2- 172.21.15.112:0/3500868097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 0x7fa410108ae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa408009ca0 tx=0x7fa40802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 -- 172.21.15.112:0/3500868097 shutdown_connections 2024-04-24T23:18:18.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 --2- 172.21.15.112:0/3500868097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 0x7fa410108ae0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 --2- 172.21.15.112:0/3500868097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa410101890 0x7fa410101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 --2- 172.21.15.112:0/3500868097 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4101006a0 0x7fa410100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 -- 172.21.15.112:0/3500868097 >> 172.21.15.112:0/3500868097 conn(0x7fa4100fbc10 msgr2=0x7fa4100fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 -- 172.21.15.112:0/3500868097 shutdown_connections 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 -- 172.21.15.112:0/3500868097 wait complete. 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 Processor -- start 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 -- start start 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4101006a0 0x7fa41006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa410101890 0x7fa41019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 0x7fa41019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa41010c440 con 0x7fa410102250 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fa41010c2c0 con 0x7fa410101890 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa417b1d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa41010c5c0 con 0x7fa4101006a0 2024-04-24T23:18:18.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa4160ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 0x7fa41019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.788+0000 7fa4160ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 0x7fa41019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52082/0 (socket says 172.21.15.112:52082) 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4160ba640 1 -- 172.21.15.112:0/277415968 learned_addr learned my addr 172.21.15.112:0/277415968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4150b8640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa410101890 0x7fa41019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4158b9640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4101006a0 0x7fa41006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4160ba640 1 -- 172.21.15.112:0/277415968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4101006a0 msgr2=0x7fa41006a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4160ba640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4101006a0 0x7fa41006a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4160ba640 1 -- 172.21.15.112:0/277415968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa410101890 msgr2=0x7fa41019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4160ba640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa410101890 0x7fa41019c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4160ba640 1 -- 172.21.15.112:0/277415968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41019f670 con 0x7fa410102250 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa4160ba640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 0x7fa41019edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa408009dc0 tx=0x7fa40802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa406ffd640 1 -- 172.21.15.112:0/277415968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40802fe40 con 0x7fa410102250 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa406ffd640 1 -- 172.21.15.112:0/277415968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fa408031df0 con 0x7fa410102250 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41019f900 con 0x7fa410102250 2024-04-24T23:18:18.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa406ffd640 1 -- 172.21.15.112:0/277415968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408046ac0 con 0x7fa410102250 2024-04-24T23:18:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.789+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41019fe50 con 0x7fa410102250 2024-04-24T23:18:18.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.790+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4101078f0 con 0x7fa410102250 2024-04-24T23:18:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.791+0000 7fa406ffd640 1 -- 172.21.15.112:0/277415968 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa408046c60 con 0x7fa410102250 2024-04-24T23:18:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.792+0000 7fa406ffd640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3fc0a2b90 0x7fa3fc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.792+0000 7fa4158b9640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3fc0a2b90 0x7fa3fc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.792+0000 7fa406ffd640 1 -- 172.21.15.112:0/277415968 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fa4080f0bf0 con 0x7fa410102250 2024-04-24T23:18:18.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.792+0000 7fa4158b9640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3fc0a2b90 0x7fa3fc0a5050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0058c0 tx=0x7fa40c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:18.798 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.796+0000 7fa406ffd640 1 -- 172.21.15.112:0/277415968 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa4080bd760 con 0x7fa410102250 2024-04-24T23:18:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.971+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa410062340 con 0x7fa410102250 2024-04-24T23:18:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.972+0000 7fa406ffd640 1 -- 172.21.15.112:0/277415968 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fa4080bd060 con 0x7fa410102250 2024-04-24T23:18:18.974 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-24T23:18:18.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3fc0a2b90 msgr2=0x7fa3fc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3fc0a2b90 0x7fa3fc0a5050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0058c0 tx=0x7fa40c005850 comp rx=0 tx=0).stop 2024-04-24T23:18:18.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 msgr2=0x7fa41019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:18.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 0x7fa41019edb0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa408009dc0 tx=0x7fa40802f840 comp rx=0 tx=0).stop 2024-04-24T23:18:18.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 shutdown_connections 2024-04-24T23:18:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fa3fc0a2b90 0x7fa3fc0a5050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa410102250 0x7fa41019edb0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa410101890 0x7fa41019c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 --2- 172.21.15.112:0/277415968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4101006a0 0x7fa41006a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 >> 172.21.15.112:0/277415968 conn(0x7fa4100fbc10 msgr2=0x7fa410104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 shutdown_connections 2024-04-24T23:18:18.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:18.974+0000 7fa417b1d640 1 -- 172.21.15.112:0/277415968 wait complete. 2024-04-24T23:18:18.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.085+0000 7f9ea1812640 1 -- 172.21.15.112:0/3889030662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c100690 msgr2=0x7f9e9c100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.085+0000 7f9ea1812640 1 --2- 172.21.15.112:0/3889030662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c100690 0x7f9e9c100ab0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009ca0 tx=0x7f9e8c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.085+0000 7f9ea1812640 1 -- 172.21.15.112:0/3889030662 shutdown_connections 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.085+0000 7f9ea1812640 1 --2- 172.21.15.112:0/3889030662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e9c102240 0x7f9e9c108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.085+0000 7f9ea1812640 1 --2- 172.21.15.112:0/3889030662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e9c101880 0x7f9e9c101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.085+0000 7f9ea1812640 1 --2- 172.21.15.112:0/3889030662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c100690 0x7f9e9c100ab0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.085+0000 7f9ea1812640 1 -- 172.21.15.112:0/3889030662 >> 172.21.15.112:0/3889030662 conn(0x7f9e9c0fbc00 msgr2=0x7f9e9c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.085+0000 7f9ea1812640 1 -- 172.21.15.112:0/3889030662 shutdown_connections 2024-04-24T23:18:19.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 -- 172.21.15.112:0/3889030662 wait complete. 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 Processor -- start 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 -- start start 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e9c100690 0x7f9e9c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e9c101880 0x7f9e9c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c102240 0x7f9e9c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c10c440 con 0x7f9e9c102240 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c10c2c0 con 0x7f9e9c101880 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9ea1812640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c10c5c0 con 0x7f9e9c100690 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.086+0000 7f9e9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c102240 0x7f9e9c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9a7fc640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e9c101880 0x7f9e9c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c102240 0x7f9e9c19f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52092/0 (socket says 172.21.15.112:52092) 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9b7fe640 1 -- 172.21.15.112:0/2833839824 learned_addr learned my addr 172.21.15.112:0/2833839824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9affd640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e9c100690 0x7f9e9c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9b7fe640 1 -- 172.21.15.112:0/2833839824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e9c100690 msgr2=0x7f9e9c197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9b7fe640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e9c100690 0x7f9e9c197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9b7fe640 1 -- 172.21.15.112:0/2833839824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e9c101880 msgr2=0x7f9e9c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9b7fe640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e9c101880 0x7f9e9c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9b7fe640 1 -- 172.21.15.112:0/2833839824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9c19f870 con 0x7f9e9c102240 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9a7fc640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e9c101880 0x7f9e9c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9e9b7fe640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c102240 0x7f9e9c19f1d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c970 tx=0x7f9e9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9ea0810640 1 -- 172.21.15.112:0/2833839824 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94007bb0 con 0x7f9e9c102240 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9ea0810640 1 -- 172.21.15.112:0/2833839824 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9e94007d50 con 0x7f9e9c102240 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9c19fb60 con 0x7f9e9c102240 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9ea0810640 1 -- 172.21.15.112:0/2833839824 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400f450 con 0x7f9e9c102240 2024-04-24T23:18:19.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.087+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9c1a00b0 con 0x7f9e9c102240 2024-04-24T23:18:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.088+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e9c1a02e0 con 0x7f9e9c102240 2024-04-24T23:18:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.089+0000 7f9ea0810640 1 -- 172.21.15.112:0/2833839824 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9e94005020 con 0x7f9e9c102240 2024-04-24T23:18:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.090+0000 7f9ea0810640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9e880a2b90 0x7f9e880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.090+0000 7f9ea0810640 1 -- 172.21.15.112:0/2833839824 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f9e940cf190 con 0x7f9e9c102240 2024-04-24T23:18:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.090+0000 7f9e9affd640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9e880a2b90 0x7f9e880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.090+0000 7f9e9affd640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9e880a2b90 0x7f9e880a5050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c031c10 tx=0x7f9e8c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:19.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.094+0000 7f9ea0810640 1 -- 172.21.15.112:0/2833839824 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9e94014030 con 0x7f9e9c102240 2024-04-24T23:18:19.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:19 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3742883796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:19 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1394710056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:19 smithi112 ceph-mon[29186]: pgmap v470: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:19.279 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:19 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/277415968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.286+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9e9c1a04b0 con 0x7f9e9c102240 2024-04-24T23:18:19.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.286+0000 7f9ea0810640 1 -- 172.21.15.112:0/2833839824 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) ==== 81+0+601 (secure 0 0 0) 0x7f9e9409bd00 con 0x7f9e9c102240 2024-04-24T23:18:19.288 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1532442772 2024-04-25T23:05:26.702079+0000 2024-04-24T23:18:19.288 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2220790491 2024-04-25T23:05:26.702079+0000 2024-04-24T23:18:19.288 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/342211680 2024-04-25T23:05:26.702079+0000 2024-04-24T23:18:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/2319978079 2024-04-25T23:05:26.702079+0000 2024-04-24T23:18:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2957535572 2024-04-25T23:05:00.376626+0000 2024-04-24T23:18:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1556121872 2024-04-25T23:05:00.376626+0000 2024-04-24T23:18:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/3348191830 2024-04-25T23:05:00.376626+0000 2024-04-24T23:18:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/3348191830 2024-04-25T23:05:00.376626+0000 2024-04-24T23:18:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4091217333 2024-04-25T23:05:00.376626+0000 2024-04-24T23:18:19.289 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/2319978079 2024-04-25T23:05:26.702079+0000 2024-04-24T23:18:19.289 INFO:teuthology.orchestra.run.smithi112.stderr:listed 10 entries 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9e880a2b90 msgr2=0x7f9e880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9e880a2b90 0x7f9e880a5050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c031c10 tx=0x7f9e8c009360 comp rx=0 tx=0).stop 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c102240 msgr2=0x7f9e9c19f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c102240 0x7f9e9c19f1d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c970 tx=0x7f9e9400ce30 comp rx=0 tx=0).stop 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 shutdown_connections 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f9e880a2b90 0x7f9e880a5050 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e9c102240 0x7f9e9c19f1d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9e9c101880 0x7f9e9c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 --2- 172.21.15.112:0/2833839824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e9c100690 0x7f9e9c197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.288+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 >> 172.21.15.112:0/2833839824 conn(0x7f9e9c0fbc00 msgr2=0x7f9e9c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:19.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.289+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 shutdown_connections 2024-04-24T23:18:19.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.289+0000 7f9ea1812640 1 -- 172.21.15.112:0/2833839824 wait complete. 2024-04-24T23:18:19.302 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-24T23:18:19.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:18:19.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-24T23:18:19.348 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-24T23:18:19.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:18:19.349 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-24T23:18:19.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:18:19.370 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-24T23:18:19.428 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-24T23:18:19.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:18:19.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:18:19.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:19 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3742883796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:19 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1394710056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:19 smithi187 ceph-mon[33292]: pgmap v470: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:19.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:19 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/277415968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:18:19.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-24T23:18:19.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:19 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3742883796' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:19 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1394710056' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:19 smithi150 ceph-mon[32247]: pgmap v470: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-24T23:18:19.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:19 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/277415968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-24T23:18:19.578 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T23:18:19.589 INFO:tasks.cephadm:Teardown begin 2024-04-24T23:18:19.589 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-24T23:18:19.589 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:18:19.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 -- 172.21.15.112:0/3975908063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688102250 msgr2=0x7fe688108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 --2- 172.21.15.112:0/3975908063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688102250 0x7fe688108ae0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe680009ca0 tx=0x7fe68002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 -- 172.21.15.112:0/3975908063 shutdown_connections 2024-04-24T23:18:19.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 --2- 172.21.15.112:0/3975908063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688102250 0x7fe688108ae0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 --2- 172.21.15.112:0/3975908063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe688101890 0x7fe688101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 --2- 172.21.15.112:0/3975908063 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6881006a0 0x7fe688100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 -- 172.21.15.112:0/3975908063 >> 172.21.15.112:0/3975908063 conn(0x7fe6880fbc30 msgr2=0x7fe6880fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 -- 172.21.15.112:0/3975908063 shutdown_connections 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.687+0000 7fe68eae7640 1 -- 172.21.15.112:0/3975908063 wait complete. 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68eae7640 1 Processor -- start 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68eae7640 1 -- start start 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68eae7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6881006a0 0x7fe688197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68eae7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688101890 0x7fe688198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68eae7640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe688102250 0x7fe68819f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68eae7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe68810c550 con 0x7fe688101890 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68eae7640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fe68810c3d0 con 0x7fe688102250 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68eae7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe68810c6d0 con 0x7fe6881006a0 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688101890 0x7fe688198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688101890 0x7fe688198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52116/0 (socket says 172.21.15.112:52116) 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 -- 172.21.15.112:0/1346183712 learned_addr learned my addr 172.21.15.112:0/1346183712 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:19.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68c883640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6881006a0 0x7fe688197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68d084640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe688102250 0x7fe68819f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 -- 172.21.15.112:0/1346183712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6881006a0 msgr2=0x7fe688197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6881006a0 0x7fe688197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 -- 172.21.15.112:0/1346183712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe688102250 msgr2=0x7fe68819f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe688102250 0x7fe68819f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 -- 172.21.15.112:0/1346183712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68819f940 con 0x7fe688101890 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe68d084640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe688102250 0x7fe68819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67ffff640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688101890 0x7fe688198270 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b4a0 tx=0x7fe67800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.688+0000 7fe67dffb640 1 -- 172.21.15.112:0/1346183712 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678014070 con 0x7fe688101890 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.689+0000 7fe67dffb640 1 -- 172.21.15.112:0/1346183712 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fe678010b10 con 0x7fe688101890 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.689+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68819fc30 con 0x7fe688101890 2024-04-24T23:18:19.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.689+0000 7fe67dffb640 1 -- 172.21.15.112:0/1346183712 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe678017d60 con 0x7fe688101890 2024-04-24T23:18:19.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.689+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6881a0130 con 0x7fe688101890 2024-04-24T23:18:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.690+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6881a0360 con 0x7fe688101890 2024-04-24T23:18:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.690+0000 7fe67dffb640 1 -- 172.21.15.112:0/1346183712 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe678017070 con 0x7fe688101890 2024-04-24T23:18:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.691+0000 7fe67dffb640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6700a2b90 0x7fe6700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.691+0000 7fe68c883640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6700a2b90 0x7fe6700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.691+0000 7fe67dffb640 1 -- 172.21.15.112:0/1346183712 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fe6780ce930 con 0x7fe688101890 2024-04-24T23:18:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.691+0000 7fe68c883640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6700a2b90 0x7fe6700a5050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe688199200 tx=0x7fe684008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:19.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.696+0000 7fe67dffb640 1 -- 172.21.15.112:0/1346183712 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe67809b4a0 con 0x7fe688101890 2024-04-24T23:18:19.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.908+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6881a0530 con 0x7fe688101890 2024-04-24T23:18:19.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.909+0000 7fe67dffb640 1 -- 172.21.15.112:0/1346183712 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11970 (secure 0 0 0) 0x7fe67809ada0 con 0x7fe688101890 2024-04-24T23:18:19.911 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:18:19.911 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:17:29.945643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24425,24424,24490,24482,24524]},"id":1}]} 2024-04-24T23:18:19.912 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 38 2024-04-24T23:18:19.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6700a2b90 msgr2=0x7fe6700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6700a2b90 0x7fe6700a5050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe688199200 tx=0x7fe684008040 comp rx=0 tx=0).stop 2024-04-24T23:18:19.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688101890 msgr2=0x7fe688198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688101890 0x7fe688198270 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe67800b4a0 tx=0x7fe67800b960 comp rx=0 tx=0).stop 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 shutdown_connections 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fe6700a2b90 0x7fe6700a5050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe688102250 0x7fe68819f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe688101890 0x7fe688198270 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 --2- 172.21.15.112:0/1346183712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6881006a0 0x7fe688197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 >> 172.21.15.112:0/1346183712 conn(0x7fe6880fbc30 msgr2=0x7fe688069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 shutdown_connections 2024-04-24T23:18:19.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:19.911+0000 7fe68eae7640 1 -- 172.21.15.112:0/1346183712 wait complete. 2024-04-24T23:18:19.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:18:20.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 -- 172.21.15.112:0/3567410225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64102230 msgr2=0x7f8e64108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 --2- 172.21.15.112:0/3567410225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64102230 0x7f8e64108ac0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009ca0 tx=0x7f8e5c02a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 -- 172.21.15.112:0/3567410225 shutdown_connections 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 --2- 172.21.15.112:0/3567410225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64102230 0x7f8e64108ac0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 --2- 172.21.15.112:0/3567410225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e64101870 0x7f8e64101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 --2- 172.21.15.112:0/3567410225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e64100680 0x7f8e64100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 -- 172.21.15.112:0/3567410225 >> 172.21.15.112:0/3567410225 conn(0x7f8e640fbbf0 msgr2=0x7f8e640fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 -- 172.21.15.112:0/3567410225 shutdown_connections 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.023+0000 7f8e6a163640 1 -- 172.21.15.112:0/3567410225 wait complete. 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e6a163640 1 Processor -- start 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e6a163640 1 -- start start 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e6a163640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e64100680 0x7f8e64195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e6a163640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64101870 0x7f8e64195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e6a163640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e64102230 0x7f8e6419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e6a163640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410b320 con 0x7f8e64101870 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e6a163640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410b1a0 con 0x7f8e64100680 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e6a163640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410b4a0 con 0x7f8e64102230 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e62ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64101870 0x7f8e64195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e62ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64101870 0x7f8e64195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52140/0 (socket says 172.21.15.112:52140) 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e62ffd640 1 -- 172.21.15.112:0/2520595458 learned_addr learned my addr 172.21.15.112:0/2520595458 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e637fe640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e64100680 0x7f8e64195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e63fff640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e64102230 0x7f8e6419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e62ffd640 1 -- 172.21.15.112:0/2520595458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e64102230 msgr2=0x7f8e6419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e62ffd640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e64102230 0x7f8e6419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e62ffd640 1 -- 172.21.15.112:0/2520595458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e64100680 msgr2=0x7f8e64195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e62ffd640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e64100680 0x7f8e64195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.024+0000 7f8e62ffd640 1 -- 172.21.15.112:0/2520595458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6419d5b0 con 0x7f8e64101870 2024-04-24T23:18:20.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.025+0000 7f8e62ffd640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64101870 0x7f8e64195f70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8e58007860 tx=0x7f8e58007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:20.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.025+0000 7f8e60ff9640 1 -- 172.21.15.112:0/2520595458 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58019020 con 0x7f8e64101870 2024-04-24T23:18:20.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.025+0000 7f8e60ff9640 1 -- 172.21.15.112:0/2520595458 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f8e58015470 con 0x7f8e64101870 2024-04-24T23:18:20.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.025+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6419d8a0 con 0x7f8e64101870 2024-04-24T23:18:20.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.025+0000 7f8e60ff9640 1 -- 172.21.15.112:0/2520595458 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58014630 con 0x7f8e64101870 2024-04-24T23:18:20.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.025+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6419ddf0 con 0x7f8e64101870 2024-04-24T23:18:20.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.026+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e64107940 con 0x7f8e64101870 2024-04-24T23:18:20.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.027+0000 7f8e60ff9640 1 -- 172.21.15.112:0/2520595458 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8e5801c020 con 0x7f8e64101870 2024-04-24T23:18:20.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.027+0000 7f8e60ff9640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8e4c0a2b90 0x7f8e4c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.028+0000 7f8e637fe640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8e4c0a2b90 0x7f8e4c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.028+0000 7f8e60ff9640 1 -- 172.21.15.112:0/2520595458 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f8e580cf900 con 0x7f8e64101870 2024-04-24T23:18:20.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.028+0000 7f8e637fe640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8e4c0a2b90 0x7f8e4c0a5050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e641016d0 tx=0x7f8e5400f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:20.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.032+0000 7f8e60ff9640 1 -- 172.21.15.112:0/2520595458 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8e58023020 con 0x7f8e64101870 2024-04-24T23:18:20.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.249+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e6402d170 con 0x7f8e64101870 2024-04-24T23:18:20.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.250+0000 7f8e60ff9640 1 -- 172.21.15.112:0/2520595458 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11970 (secure 0 0 0) 0x7f8e5809c470 con 0x7f8e64101870 2024-04-24T23:18:20.252 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:18:20.252 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14556,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/4200698623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":4200698623},{"type":"v1","addr":"172.21.15.112:6835","nonce":4200698623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14568,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3896751620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":3896751620},{"type":"v1","addr":"172.21.15.112:6837","nonce":3896751620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-24T23:12:27.330443+0000","modified":"2024-04-24T23:17:29.945643+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24490,"mds_2":24482,"mds_3":24425,"mds_4":24424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6839/2592086011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":2592086011},{"type":"v1","addr":"172.21.15.112:6839","nonce":2592086011}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.112:6841/3500837168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":3500837168},{"type":"v1","addr":"172.21.15.112:6841","nonce":3500837168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24424":{"gid":24424,"name":"l","rank":4,"incarnation":25,"state":"up:active","state_seq":18,"addr":"172.21.15.187:6833/1998066775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1998066775},{"type":"v1","addr":"172.21.15.187:6833","nonce":1998066775}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24425":{"gid":24425,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":17,"addr":"172.21.15.150:6833/68455606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":68455606},{"type":"v1","addr":"172.21.15.150:6833","nonce":68455606}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.150:6835/2076270934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":2076270934},{"type":"v1","addr":"172.21.15.150:6835","nonce":2076270934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/1908796622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":1908796622},{"type":"v1","addr":"172.21.15.187:6835","nonce":1908796622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/35756805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":35756805},{"type":"v1","addr":"172.21.15.187:6837","nonce":35756805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.150:6837/3592489674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6836","nonce":3592489674},{"type":"v1","addr":"172.21.15.150:6837","nonce":3592489674}]},"join_fscid":-1,"export_targets":[0,1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.187:6839/3343164580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":3343164580},{"type":"v1","addr":"172.21.15.187:6839","nonce":3343164580}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6839/748900426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6838","nonce":748900426},{"type":"v1","addr":"172.21.15.150:6839","nonce":748900426}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24425,24424,24490,24482,24524]},"id":1}]} 2024-04-24T23:18:20.253 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 38 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8e4c0a2b90 msgr2=0x7f8e4c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8e4c0a2b90 0x7f8e4c0a5050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e641016d0 tx=0x7f8e5400f040 comp rx=0 tx=0).stop 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64101870 msgr2=0x7f8e64195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64101870 0x7f8e64195f70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8e58007860 tx=0x7f8e58007d20 comp rx=0 tx=0).stop 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 shutdown_connections 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f8e4c0a2b90 0x7f8e4c0a5050 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e64102230 0x7f8e6419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e64101870 0x7f8e64195f70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 --2- 172.21.15.112:0/2520595458 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8e64100680 0x7f8e64195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.252+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 >> 172.21.15.112:0/2520595458 conn(0x7f8e640fbbf0 msgr2=0x7f8e64104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.253+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 shutdown_connections 2024-04-24T23:18:20.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.253+0000 7f8e6a163640 1 -- 172.21.15.112:0/2520595458 wait complete. 2024-04-24T23:18:20.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2833839824' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:18:20.264 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/1346183712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:20.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.360+0000 7f906e5c1640 1 -- 172.21.15.112:0/1380251587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068100680 msgr2=0x7f9068100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.360+0000 7f906e5c1640 1 --2- 172.21.15.112:0/1380251587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068100680 0x7f9068100aa0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9058009900 tx=0x7f905802a410 comp rx=0 tx=0).stop 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.360+0000 7f906e5c1640 1 -- 172.21.15.112:0/1380251587 shutdown_connections 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.360+0000 7f906e5c1640 1 --2- 172.21.15.112:0/1380251587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068102230 0x7f9068108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.360+0000 7f906e5c1640 1 --2- 172.21.15.112:0/1380251587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9068101870 0x7f9068101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.360+0000 7f906e5c1640 1 --2- 172.21.15.112:0/1380251587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068100680 0x7f9068100aa0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.360+0000 7f906e5c1640 1 -- 172.21.15.112:0/1380251587 >> 172.21.15.112:0/1380251587 conn(0x7f90680fbbf0 msgr2=0x7f90680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 -- 172.21.15.112:0/1380251587 shutdown_connections 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 -- 172.21.15.112:0/1380251587 wait complete. 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 Processor -- start 2024-04-24T23:18:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 -- start start 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068100680 0x7f9068069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068101870 0x7f906819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9068102230 0x7f90681a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f906810c430 con 0x7f9068101870 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f906810c2b0 con 0x7f9068100680 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f906e5c1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f906810c5b0 con 0x7f9068102230 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f90677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068101870 0x7f906819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f90677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068101870 0x7f906819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52146/0 (socket says 172.21.15.112:52146) 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.361+0000 7f90677fe640 1 -- 172.21.15.112:0/3948202312 learned_addr learned my addr 172.21.15.112:0/3948202312 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f906cb5e640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9068102230 0x7f90681a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f9067fff640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068100680 0x7f9068069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90677fe640 1 -- 172.21.15.112:0/3948202312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9068102230 msgr2=0x7f90681a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90677fe640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9068102230 0x7f90681a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90677fe640 1 -- 172.21.15.112:0/3948202312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068100680 msgr2=0x7f9068069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90677fe640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068100680 0x7f9068069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90677fe640 1 -- 172.21.15.112:0/3948202312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90681a19d0 con 0x7f9068101870 2024-04-24T23:18:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90677fe640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068101870 0x7f906819ec90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f905000d770 tx=0x7f905000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90657fa640 1 -- 172.21.15.112:0/3948202312 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050017070 con 0x7f9068101870 2024-04-24T23:18:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90657fa640 1 -- 172.21.15.112:0/3948202312 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f9050005520 con 0x7f9068101870 2024-04-24T23:18:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90681a1c60 con 0x7f9068101870 2024-04-24T23:18:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f90657fa640 1 -- 172.21.15.112:0/3948202312 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050010460 con 0x7f9068101870 2024-04-24T23:18:20.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.362+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a2160 con 0x7f9068101870 2024-04-24T23:18:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.363+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90681079a0 con 0x7f9068101870 2024-04-24T23:18:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.364+0000 7f90657fa640 1 -- 172.21.15.112:0/3948202312 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9050004ea0 con 0x7f9068101870 2024-04-24T23:18:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.365+0000 7f90657fa640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f90540a2bc0 0x7f90540a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.365+0000 7f9067fff640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f90540a2bc0 0x7f90540a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.365+0000 7f90657fa640 1 -- 172.21.15.112:0/3948202312 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7f90500ceef0 con 0x7f9068101870 2024-04-24T23:18:20.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.365+0000 7f9067fff640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f90540a2bc0 0x7f90540a5080 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9058006010 tx=0x7f9058005be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.369+0000 7f90657fa640 1 -- 172.21.15.112:0/3948202312 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f905009ba60 con 0x7f9068101870 2024-04-24T23:18:20.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2833839824' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:18:20.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/1346183712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:20.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.547+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9068062340 con 0x7f9068101870 2024-04-24T23:18:20.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.548+0000 7f90657fa640 1 -- 172.21.15.112:0/3948202312 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24950 (secure 0 0 0) 0x7f905009b360 con 0x7f9068101870 2024-04-24T23:18:20.550 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-24T23:18:20.550 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":94,"fsid":"e62ba378-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:04:28.856850+0000","modified":"2024-04-24T23:14:28.925442+0000","last_up_change":"2024-04-24T23:11:37.631250+0000","last_in_change":"2024-04-24T23:11:18.227619+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:07:51.018846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-24T23:12:25.486401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-24T23:12:26.512604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-24T23:12:30.239863+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c7d3b0e-ce0f-4b59-955f-98303b9bd863","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6803","nonce":3410548369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6805","nonce":3410548369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6809","nonce":3410548369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3410548369},{"type":"v1","addr":"172.21.15.112:6807","nonce":3410548369}]},"public_addr":"172.21.15.112:6803/3410548369","cluster_addr":"172.21.15.112:6805/3410548369","heartbeat_back_addr":"172.21.15.112:6809/3410548369","heartbeat_front_addr":"172.21.15.112:6807/3410548369","state":["exists","up"]},{"osd":1,"uuid":"a46ec67e-696c-497b-a0f4-9a38bae79a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6801","nonce":459871910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6803","nonce":459871910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6807","nonce":459871910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":459871910},{"type":"v1","addr":"172.21.15.150:6805","nonce":459871910}]},"public_addr":"172.21.15.150:6801/459871910","cluster_addr":"172.21.15.150:6803/459871910","heartbeat_back_addr":"172.21.15.150:6807/459871910","heartbeat_front_addr":"172.21.15.150:6805/459871910","state":["exists","up"]},{"osd":2,"uuid":"4899db4a-7c50-4d57-bce9-50d0e840ca6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6801","nonce":3027054636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6803","nonce":3027054636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6807","nonce":3027054636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":3027054636},{"type":"v1","addr":"172.21.15.187:6805","nonce":3027054636}]},"public_addr":"172.21.15.187:6801/3027054636","cluster_addr":"172.21.15.187:6803/3027054636","heartbeat_back_addr":"172.21.15.187:6807/3027054636","heartbeat_front_addr":"172.21.15.187:6805/3027054636","state":["exists","up"]},{"osd":3,"uuid":"6c500d4f-a92d-489a-98c8-758763754114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6811","nonce":1421446388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6813","nonce":1421446388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6817","nonce":1421446388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1421446388},{"type":"v1","addr":"172.21.15.112:6815","nonce":1421446388}]},"public_addr":"172.21.15.112:6811/1421446388","cluster_addr":"172.21.15.112:6813/1421446388","heartbeat_back_addr":"172.21.15.112:6817/1421446388","heartbeat_front_addr":"172.21.15.112:6815/1421446388","state":["exists","up"]},{"osd":4,"uuid":"57a393a7-46ea-440d-b990-315d5806e675","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6809","nonce":3997288444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6810","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6811","nonce":3997288444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6814","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6815","nonce":3997288444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":3997288444},{"type":"v1","addr":"172.21.15.150:6813","nonce":3997288444}]},"public_addr":"172.21.15.150:6809/3997288444","cluster_addr":"172.21.15.150:6811/3997288444","heartbeat_back_addr":"172.21.15.150:6815/3997288444","heartbeat_front_addr":"172.21.15.150:6813/3997288444","state":["exists","up"]},{"osd":5,"uuid":"905c0bbc-34be-4035-9799-e3385ea78043","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6809","nonce":1092655909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6811","nonce":1092655909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6815","nonce":1092655909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1092655909},{"type":"v1","addr":"172.21.15.187:6813","nonce":1092655909}]},"public_addr":"172.21.15.187:6809/1092655909","cluster_addr":"172.21.15.187:6811/1092655909","heartbeat_back_addr":"172.21.15.187:6815/1092655909","heartbeat_front_addr":"172.21.15.187:6813/1092655909","state":["exists","up"]},{"osd":6,"uuid":"14b8b9f0-b530-4914-a7c6-d3b8b9fa6002","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6819","nonce":1354124649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6821","nonce":1354124649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6825","nonce":1354124649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1354124649},{"type":"v1","addr":"172.21.15.112:6823","nonce":1354124649}]},"public_addr":"172.21.15.112:6819/1354124649","cluster_addr":"172.21.15.112:6821/1354124649","heartbeat_back_addr":"172.21.15.112:6825/1354124649","heartbeat_front_addr":"172.21.15.112:6823/1354124649","state":["exists","up"]},{"osd":7,"uuid":"aacba960-f7c4-41cd-8ee0-cac0fba62c5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6817","nonce":591791108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6818","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6819","nonce":591791108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6823","nonce":591791108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":591791108},{"type":"v1","addr":"172.21.15.150:6821","nonce":591791108}]},"public_addr":"172.21.15.150:6817/591791108","cluster_addr":"172.21.15.150:6819/591791108","heartbeat_back_addr":"172.21.15.150:6823/591791108","heartbeat_front_addr":"172.21.15.150:6821/591791108","state":["exists","up"]},{"osd":8,"uuid":"c907152a-2f78-4da1-8c5a-853706d3c59f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6817","nonce":3574382245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6819","nonce":3574382245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6823","nonce":3574382245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3574382245},{"type":"v1","addr":"172.21.15.187:6821","nonce":3574382245}]},"public_addr":"172.21.15.187:6817/3574382245","cluster_addr":"172.21.15.187:6819/3574382245","heartbeat_back_addr":"172.21.15.187:6823/3574382245","heartbeat_front_addr":"172.21.15.187:6821/3574382245","state":["exists","up"]},{"osd":9,"uuid":"e6e8af2d-787c-472a-973b-da754768b4e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6827","nonce":3646247852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6829","nonce":3646247852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6833","nonce":3646247852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3646247852},{"type":"v1","addr":"172.21.15.112:6831","nonce":3646247852}]},"public_addr":"172.21.15.112:6827/3646247852","cluster_addr":"172.21.15.112:6829/3646247852","heartbeat_back_addr":"172.21.15.112:6833/3646247852","heartbeat_front_addr":"172.21.15.112:6831/3646247852","state":["exists","up"]},{"osd":10,"uuid":"0114d00e-ffc8-4e0d-b79d-c87a604b5cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6825","nonce":599401021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6827","nonce":599401021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6831","nonce":599401021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":599401021},{"type":"v1","addr":"172.21.15.150:6829","nonce":599401021}]},"public_addr":"172.21.15.150:6825/599401021","cluster_addr":"172.21.15.150:6827/599401021","heartbeat_back_addr":"172.21.15.150:6831/599401021","heartbeat_front_addr":"172.21.15.150:6829/599401021","state":["exists","up"]},{"osd":11,"uuid":"971bddea-134b-4d7e-9ae6-770a9129fdb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6825","nonce":3702998666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6827","nonce":3702998666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6831","nonce":3702998666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":3702998666},{"type":"v1","addr":"172.21.15.187:6829","nonce":3702998666}]},"public_addr":"172.21.15.187:6825/3702998666","cluster_addr":"172.21.15.187:6827/3702998666","heartbeat_back_addr":"172.21.15.187:6831/3702998666","heartbeat_front_addr":"172.21.15.187:6829/3702998666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:00.483799+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:24.429761+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:47.775342+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:12.545176+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:37.689364+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:02.422266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:27.554177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:53.920261+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:18.895833+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:10:44.963278+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:10.859253+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:11:36.280188+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/1532442772":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2220790491":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/342211680":"2024-04-25T23:05:26.702079+0000","172.21.15.112:6801/2319978079":"2024-04-25T23:05:26.702079+0000","172.21.15.112:0/2957535572":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/1556121872":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6801/3348191830":"2024-04-25T23:05:00.376626+0000","172.21.15.112:0/4091217333":"2024-04-25T23:05:00.376626+0000","172.21.15.112:6800/2319978079":"2024-04-25T23:05:26.702079+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:18:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f90540a2bc0 msgr2=0x7f90540a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f90540a2bc0 0x7f90540a5080 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9058006010 tx=0x7f9058005be0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068101870 msgr2=0x7f906819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068101870 0x7f906819ec90 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f905000d770 tx=0x7f905000dc30 comp rx=0 tx=0).stop 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 shutdown_connections 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f90540a2bc0 0x7f90540a5080 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9068102230 0x7f90681a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9068101870 0x7f906819ec90 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 --2- 172.21.15.112:0/3948202312 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9068100680 0x7f9068069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 >> 172.21.15.112:0/3948202312 conn(0x7f90680fbbf0 msgr2=0x7f9068104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 shutdown_connections 2024-04-24T23:18:20.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.550+0000 7f906e5c1640 1 -- 172.21.15.112:0/3948202312 wait complete. 2024-04-24T23:18:20.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-24T23:18:20.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2833839824' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:18:20.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/1346183712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 -- 172.21.15.112:0/2829937552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120103340 msgr2=0x7fb120109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 --2- 172.21.15.112:0/2829937552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120103340 0x7fb120109bd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb118009ca0 tx=0x7fb11802a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 -- 172.21.15.112:0/2829937552 shutdown_connections 2024-04-24T23:18:20.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 --2- 172.21.15.112:0/2829937552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120103340 0x7fb120109bd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 --2- 172.21.15.112:0/2829937552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb120102980 0x7fb120102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 --2- 172.21.15.112:0/2829937552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb120101790 0x7fb120101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 -- 172.21.15.112:0/2829937552 >> 172.21.15.112:0/2829937552 conn(0x7fb1200fcd20 msgr2=0x7fb1200ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 -- 172.21.15.112:0/2829937552 shutdown_connections 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 -- 172.21.15.112:0/2829937552 wait complete. 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 Processor -- start 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 -- start start 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120101790 0x7fb120197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.661+0000 7fb127cb3640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb120102980 0x7fb120197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb127cb3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb120103340 0x7fb12019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb127cb3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d530 con 0x7fb120101790 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb127cb3640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d3b0 con 0x7fb120102980 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb127cb3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb12010d6b0 con 0x7fb120103340 2024-04-24T23:18:20.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120101790 0x7fb120197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120101790 0x7fb120197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50880/0 (socket says 172.21.15.112:50880) 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 -- 172.21.15.112:0/985963550 learned_addr learned my addr 172.21.15.112:0/985963550 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb12524e640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb120102980 0x7fb120197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 -- 172.21.15.112:0/985963550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb120103340 msgr2=0x7fb12019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb120103340 0x7fb12019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 -- 172.21.15.112:0/985963550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb120102980 msgr2=0x7fb120197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb120102980 0x7fb120197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 -- 172.21.15.112:0/985963550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12019f6a0 con 0x7fb120101790 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb125a4f640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120101790 0x7fb120197a80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00c970 tx=0x7fb11c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb112ffd640 1 -- 172.21.15.112:0/985963550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c007c10 con 0x7fb120101790 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb112ffd640 1 -- 172.21.15.112:0/985963550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7fb11c007db0 con 0x7fb120101790 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb112ffd640 1 -- 172.21.15.112:0/985963550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c00fa40 con 0x7fb120101790 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12019f990 con 0x7fb120101790 2024-04-24T23:18:20.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.662+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12019fee0 con 0x7fb120101790 2024-04-24T23:18:20.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.663+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1201089e0 con 0x7fb120101790 2024-04-24T23:18:20.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.664+0000 7fb112ffd640 1 -- 172.21.15.112:0/985963550 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb11c00fbe0 con 0x7fb120101790 2024-04-24T23:18:20.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.665+0000 7fb112ffd640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb10c0a2b90 0x7fb10c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:20.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.665+0000 7fb12524e640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb10c0a2b90 0x7fb10c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:20.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.665+0000 7fb112ffd640 1 -- 172.21.15.112:0/985963550 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9237+0+0 (secure 0 0 0) 0x7fb11c0cfae0 con 0x7fb120101790 2024-04-24T23:18:20.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.666+0000 7fb12524e640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb10c0a2b90 0x7fb10c0a5050 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb120198f90 tx=0x7fb11400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:20.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.669+0000 7fb112ffd640 1 -- 172.21.15.112:0/985963550 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb11c09c770 con 0x7fb120101790 2024-04-24T23:18:20.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.881+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb12002d420 con 0x7fb120101790 2024-04-24T23:18:20.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.883+0000 7fb112ffd640 1 -- 172.21.15.112:0/985963550 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7fb11c017e80 con 0x7fb120101790 2024-04-24T23:18:20.886 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-24T23:18:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb10c0a2b90 msgr2=0x7fb10c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb10c0a2b90 0x7fb10c0a5050 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb120198f90 tx=0x7fb11400a4c0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120101790 msgr2=0x7fb120197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120101790 0x7fb120197a80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00c970 tx=0x7fb11c00ce30 comp rx=0 tx=0).stop 2024-04-24T23:18:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 shutdown_connections 2024-04-24T23:18:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7fb10c0a2b90 0x7fb10c0a5050 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb120103340 0x7fb12019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb120102980 0x7fb120197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 --2- 172.21.15.112:0/985963550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb120101790 0x7fb120197a80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.887+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 >> 172.21.15.112:0/985963550 conn(0x7fb1200fcd20 msgr2=0x7fb120105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.888+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 shutdown_connections 2024-04-24T23:18:20.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:20.888+0000 7fb127cb3640 1 -- 172.21.15.112:0/985963550 wait complete. 2024-04-24T23:18:20.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-24T23:18:21.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 -- 172.21.15.112:0/3812455074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c103340 msgr2=0x7ff00c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 --2- 172.21.15.112:0/3812455074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c103340 0x7ff00c109bd0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff004009900 tx=0x7ff00402a410 comp rx=0 tx=0).stop 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 -- 172.21.15.112:0/3812455074 shutdown_connections 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 --2- 172.21.15.112:0/3812455074 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c103340 0x7ff00c109bd0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 --2- 172.21.15.112:0/3812455074 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c102980 0x7ff00c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 --2- 172.21.15.112:0/3812455074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c101790 0x7ff00c101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 -- 172.21.15.112:0/3812455074 >> 172.21.15.112:0/3812455074 conn(0x7ff00c0fcd20 msgr2=0x7ff00c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 -- 172.21.15.112:0/3812455074 shutdown_connections 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 -- 172.21.15.112:0/3812455074 wait complete. 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 Processor -- start 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.010+0000 7ff01205d640 1 -- start start 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff01205d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c101790 0x7ff00c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff01205d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c102980 0x7ff00c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:21.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff01205d640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c103340 0x7ff00c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff01205d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d540 con 0x7ff00c102980 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff01205d640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d3c0 con 0x7ff00c103340 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff01205d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff00c10d6c0 con 0x7ff00c101790 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c102980 0x7ff00c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c102980 0x7ff00c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50932/0 (socket says 172.21.15.112:50932) 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 -- 172.21.15.112:0/3591768556 learned_addr learned my addr 172.21.15.112:0/3591768556 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00b7fe640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c101790 0x7ff00c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00bfff640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c103340 0x7ff00c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 -- 172.21.15.112:0/3591768556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c101790 msgr2=0x7ff00c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c101790 0x7ff00c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 -- 172.21.15.112:0/3591768556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c103340 msgr2=0x7ff00c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c103340 0x7ff00c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 -- 172.21.15.112:0/3591768556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00c19f6f0 con 0x7ff00c102980 2024-04-24T23:18:21.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff00affd640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c102980 0x7ff00c198010 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff00000cce0 tx=0x7ff00000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff008ff9640 1 -- 172.21.15.112:0/3591768556 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00000f650 con 0x7ff00c102980 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00c19f9e0 con 0x7ff00c102980 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff008ff9640 1 -- 172.21.15.112:0/3591768556 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff00000fcf0 con 0x7ff00c102980 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff008ff9640 1 -- 172.21.15.112:0/3591768556 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000018d20 con 0x7ff00c102980 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.011+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00c19ff80 con 0x7ff00c102980 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.012+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff00c108a50 con 0x7ff00c102980 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.013+0000 7ff008ff9640 1 -- 172.21.15.112:0/3591768556 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff00000f7f0 con 0x7ff00c102980 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.014+0000 7ff008ff9640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.014+0000 7ff00b7fe640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.014+0000 7ff008ff9640 1 -- 172.21.15.112:0/3591768556 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9667+0+0 (secure 0 0 0) 0x7ff0000cfb40 con 0x7ff00c102980 2024-04-24T23:18:21.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.014+0000 7ff00b7fe640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff00c1027e0 tx=0x7feffc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:21.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.018+0000 7ff008ff9640 1 -- 172.21.15.112:0/3591768556 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff000014030 con 0x7ff00c102980 2024-04-24T23:18:21.183 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.183 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.183 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-mds[62635]: -- 172.21.15.150:0/2766198039 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654f5919a40 con 0x5654f58ec800 2024-04-24T23:18:21.183 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: starting mds.b at 2024-04-24T23:18:21.184 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e[62000]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.184 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e[62000]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.184 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-mds[62004]: -- 172.21.15.150:0/4052344279 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a6c879a40 con 0x560a6c84c800 2024-04-24T23:18:21.184 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e[62000]: starting mds.e at 2024-04-24T23:18:21.185 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k[61682]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.185 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k[61682]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.185 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-mds[61686]: -- 172.21.15.150:0/2225503773 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad92f87a40 con 0x55ad92f5b000 2024-04-24T23:18:21.185 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k[61682]: starting mds.k at 2024-04-24T23:18:21.185 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h[62315]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.185 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h[62315]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.185 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-mds[62319]: -- 172.21.15.150:0/3619469210 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651432e7a40 con 0x5651432ba800 2024-04-24T23:18:21.185 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:18:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h[62315]: starting mds.h at 2024-04-24T23:18:21.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2520595458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:21.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3948202312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:18:21.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/985963550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:18:21.186 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:18:21.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:18:21.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: pgmap v471: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T23:18:21.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/985963550' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:18:21.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:18:21.187 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:21 smithi150 ceph-mon[32247]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T23:18:21.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.232+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff00c062340 con 0x7ff00c102980 2024-04-24T23:18:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2520595458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3948202312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:18:21.236 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/985963550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: pgmap v471: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/985963550' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:21 smithi187 ceph-mon[33292]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c[63570]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c[63570]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-mds[63574]: -- 172.21.15.187:0/922874073 <== mon.2 v2:172.21.15.150:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56432f49fa40 con 0x56432f472800 2024-04-24T23:18:21.237 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c[63570]: starting mds.c at 2024-04-24T23:18:21.238 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f[63267]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.238 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f[63267]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.238 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-mds[63271]: -- 172.21.15.187:0/2314800915 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562aedc3da40 con 0x562aedc10800 2024-04-24T23:18:21.238 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f[63267]: starting mds.f at 2024-04-24T23:18:21.238 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-i[63885]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.238 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-i[63885]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.239 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-mds[63889]: -- 172.21.15.187:0/1392698839 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8c9d75a40 con 0x55a8c9d49000 2024-04-24T23:18:21.239 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-i[63885]: starting mds.i at 2024-04-24T23:18:21.239 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l[62963]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.239 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l[62963]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.239 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-mds[62967]: -- 172.21.15.187:0/3646037310 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56479f8a3a40 con 0x56479f877000 2024-04-24T23:18:21.239 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:18:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l[62963]: starting mds.l at 2024-04-24T23:18:21.279 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a[71721]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.279 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a[71721]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.279 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-mds[71727]: -- 172.21.15.112:0/3041835778 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eba19a1a40 con 0x55eba1974c00 2024-04-24T23:18:21.279 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a[71721]: starting mds.a at 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g[71363]: ignoring --setuser ceph since I am not root 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g[71363]: ignoring --setgroup ceph since I am not root 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-mds[71367]: -- 172.21.15.112:0/657902206 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fae39dfa40 con 0x55fae39b3000 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g[71363]: starting mds.g at 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a[29182]: 2024-04-24T23:18:20.881+0000 7fd10414e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2520595458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3948202312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/985963550' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-24T23:18:21.280 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:18:21.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:18:21.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: pgmap v471: 129 pgs: 129 active+clean; 109 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-24T23:18:21.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/985963550' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-24T23:18:21.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: osdmap e95: 12 total, 12 up, 12 in 2024-04-24T23:18:21.281 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:21 smithi112 ceph-mon[29186]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-24T23:18:21.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.885+0000 7ff008ff9640 1 -- 172.21.15.112:0/3591768556 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7ff00009c500 con 0x7ff00c102980 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 msgr2=0x7feff40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff00c1027e0 tx=0x7feffc005f90 comp rx=0 tx=0).stop 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c102980 msgr2=0x7ff00c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c102980 0x7ff00c198010 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff00000cce0 tx=0x7ff00000a620 comp rx=0 tx=0).stop 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 shutdown_connections 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7feff40a2b90 0x7feff40a5050 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff00c103340 0x7ff00c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff00c102980 0x7ff00c198010 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 --2- 172.21.15.112:0/3591768556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff00c101790 0x7ff00c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.888+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 >> 172.21.15.112:0/3591768556 conn(0x7ff00c0fcd20 msgr2=0x7ff00c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.889+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 shutdown_connections 2024-04-24T23:18:21.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:21.889+0000 7ff01205d640 1 -- 172.21.15.112:0/3591768556 wait complete. 2024-04-24T23:18:21.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-24T23:18:22.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 -- 172.21.15.112:0/3799089281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff68102230 msgr2=0x7eff68108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:22.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 --2- 172.21.15.112:0/3799089281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff68102230 0x7eff68108ac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7eff60009ca0 tx=0x7eff6002a8c0 comp rx=0 tx=0).stop 2024-04-24T23:18:22.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 -- 172.21.15.112:0/3799089281 shutdown_connections 2024-04-24T23:18:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 --2- 172.21.15.112:0/3799089281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff68102230 0x7eff68108ac0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 --2- 172.21.15.112:0/3799089281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68101870 0x7eff68101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 --2- 172.21.15.112:0/3799089281 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff68100680 0x7eff68100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 -- 172.21.15.112:0/3799089281 >> 172.21.15.112:0/3799089281 conn(0x7eff680fbbf0 msgr2=0x7eff680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 -- 172.21.15.112:0/3799089281 shutdown_connections 2024-04-24T23:18:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 -- 172.21.15.112:0/3799089281 wait complete. 2024-04-24T23:18:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.015+0000 7eff6e563640 1 Processor -- start 2024-04-24T23:18:22.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff6e563640 1 -- start start 2024-04-24T23:18:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff6e563640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff68100680 0x7eff68069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff6e563640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff68101870 0x7eff6819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff6e563640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68102230 0x7eff6819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff6e563640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810c430 con 0x7eff68101870 2024-04-24T23:18:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff6e563640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810c2b0 con 0x7eff68102230 2024-04-24T23:18:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff6e563640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810c5b0 con 0x7eff68100680 2024-04-24T23:18:22.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff68101870 0x7eff6819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff68101870 0x7eff6819c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50952/0 (socket says 172.21.15.112:50952) 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 -- 172.21.15.112:0/2042012486 learned_addr learned my addr 172.21.15.112:0/2042012486 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff6cb00640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68102230 0x7eff6819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff67fff640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff68100680 0x7eff68069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 -- 172.21.15.112:0/2042012486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff68100680 msgr2=0x7eff68069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff68100680 0x7eff68069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 -- 172.21.15.112:0/2042012486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68102230 msgr2=0x7eff6819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68102230 0x7eff6819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 -- 172.21.15.112:0/2042012486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6819f790 con 0x7eff68101870 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.016+0000 7eff677fe640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff68101870 0x7eff6819c860 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e9f0 tx=0x7eff5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.017+0000 7eff657fa640 1 -- 172.21.15.112:0/2042012486 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00cd80 con 0x7eff68101870 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.017+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6819fa20 con 0x7eff68101870 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.017+0000 7eff657fa640 1 -- 172.21.15.112:0/2042012486 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7eff5c005520 con 0x7eff68101870 2024-04-24T23:18:22.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.017+0000 7eff657fa640 1 -- 172.21.15.112:0/2042012486 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c010690 con 0x7eff68101870 2024-04-24T23:18:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.017+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6819ff70 con 0x7eff68101870 2024-04-24T23:18:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.018+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff68107940 con 0x7eff68101870 2024-04-24T23:18:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.019+0000 7eff657fa640 1 -- 172.21.15.112:0/2042012486 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7eff5c0046c0 con 0x7eff68101870 2024-04-24T23:18:22.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.019+0000 7eff657fa640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff500a2b90 0x7eff500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:22.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.019+0000 7eff67fff640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff500a2b90 0x7eff500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:22.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.019+0000 7eff657fa640 1 -- 172.21.15.112:0/2042012486 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9667+0+0 (secure 0 0 0) 0x7eff5c0cf9f0 con 0x7eff68101870 2024-04-24T23:18:22.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.020+0000 7eff67fff640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff500a2b90 0x7eff500a5050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7eff580095f0 tx=0x7eff58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:22.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.024+0000 7eff657fa640 1 -- 172.21.15.112:0/2042012486 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7eff5c09c3b0 con 0x7eff68101870 2024-04-24T23:18:22.187 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3591768556' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: Cluster is now healthy 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/3591768556' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.112:6838/3252544245,v1:172.21.15.112:6839/3252544245] up:boot 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6832/3569763766,v1:172.21.15.187:6833/3569763766] up:boot 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.112:6840/2894058226,v1:172.21.15.112:6841/2894058226] up:boot 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6832/284044546,v1:172.21.15.150:6834/284044546] up:boot 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6833/2105032530,v1:172.21.15.150:6835/2105032530] up:boot 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6834/719924478,v1:172.21.15.187:6835/719924478] up:boot 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6836/3690567470,v1:172.21.15.187:6837/3690567470] up:boot 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.187:6838/922584260,v1:172.21.15.187:6839/922584260] up:boot 2024-04-24T23:18:22.188 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6836/2712212903,v1:172.21.15.150:6838/2712212903] up:boot 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: mds.? [v2:172.21.15.150:6837/3294566427,v1:172.21.15.150:6839/3294566427] up:boot 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: fsmap 12 up:standby 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:18:22.189 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:22 smithi112 ceph-mon[29186]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:18:22.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:22.210+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7eff68062340 con 0x7eff68101870 2024-04-24T23:18:22.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3591768556' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:18:22.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:18:22.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:18:22.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: Cluster is now healthy 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/3591768556' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.112:6838/3252544245,v1:172.21.15.112:6839/3252544245] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6832/3569763766,v1:172.21.15.187:6833/3569763766] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.112:6840/2894058226,v1:172.21.15.112:6841/2894058226] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6832/284044546,v1:172.21.15.150:6834/284044546] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6833/2105032530,v1:172.21.15.150:6835/2105032530] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6834/719924478,v1:172.21.15.187:6835/719924478] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6836/3690567470,v1:172.21.15.187:6837/3690567470] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.187:6838/922584260,v1:172.21.15.187:6839/922584260] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6836/2712212903,v1:172.21.15.150:6838/2712212903] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: mds.? [v2:172.21.15.150:6837/3294566427,v1:172.21.15.150:6839/3294566427] up:boot 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: fsmap 12 up:standby 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:18:22.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:18:22.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:18:22.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:18:22.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:18:22.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:18:22.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:18:22.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:18:22.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:18:22.488 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:22 smithi187 ceph-mon[33292]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3591768556' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: Cluster is now healthy 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/3591768556' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.112:6838/3252544245,v1:172.21.15.112:6839/3252544245] up:boot 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6832/3569763766,v1:172.21.15.187:6833/3569763766] up:boot 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.112:6840/2894058226,v1:172.21.15.112:6841/2894058226] up:boot 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6832/284044546,v1:172.21.15.150:6834/284044546] up:boot 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6833/2105032530,v1:172.21.15.150:6835/2105032530] up:boot 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6834/719924478,v1:172.21.15.187:6835/719924478] up:boot 2024-04-24T23:18:22.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6836/3690567470,v1:172.21.15.187:6837/3690567470] up:boot 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.187:6838/922584260,v1:172.21.15.187:6839/922584260] up:boot 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6836/2712212903,v1:172.21.15.150:6838/2712212903] up:boot 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: mds.? [v2:172.21.15.150:6837/3294566427,v1:172.21.15.150:6839/3294566427] up:boot 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: fsmap 12 up:standby 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-24T23:18:22.570 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-24T23:18:22.571 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:22 smithi150 ceph-mon[32247]: from='mgr.14150 172.21.15.112:0/2803439816' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-24T23:18:23.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.114+0000 7eff657fa640 1 -- 172.21.15.112:0/2042012486 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7eff5c09bcb0 con 0x7eff68101870 2024-04-24T23:18:23.117 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-04-24T23:18:23.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.117+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff500a2b90 msgr2=0x7eff500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:23.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.117+0000 7eff6e563640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff500a2b90 0x7eff500a5050 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7eff580095f0 tx=0x7eff58005f90 comp rx=0 tx=0).stop 2024-04-24T23:18:23.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.117+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff68101870 msgr2=0x7eff6819c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:23.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.117+0000 7eff6e563640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff68101870 0x7eff6819c860 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00e9f0 tx=0x7eff5c00edf0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.118+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 shutdown_connections 2024-04-24T23:18:23.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.118+0000 7eff6e563640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7eff500a2b90 0x7eff500a5050 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.118+0000 7eff6e563640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68102230 0x7eff6819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.118+0000 7eff6e563640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7eff68101870 0x7eff6819c860 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.118+0000 7eff6e563640 1 --2- 172.21.15.112:0/2042012486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff68100680 0x7eff68069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.118+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 >> 172.21.15.112:0/2042012486 conn(0x7eff680fbbf0 msgr2=0x7eff68104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:23.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.118+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 shutdown_connections 2024-04-24T23:18:23.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.118+0000 7eff6e563640 1 -- 172.21.15.112:0/2042012486 wait complete. 2024-04-24T23:18:23.133 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-24T23:18:23.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.233+0000 7f4348b4c640 1 -- 172.21.15.112:0/1251242724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 msgr2=0x7f43441013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:23.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.233+0000 7f4348b4c640 1 --2- 172.21.15.112:0/1251242724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 0x7f43441013e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f433800aa10 tx=0x7f433802b720 comp rx=0 tx=0).stop 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.233+0000 7f4348b4c640 1 -- 172.21.15.112:0/1251242724 shutdown_connections 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.233+0000 7f4348b4c640 1 --2- 172.21.15.112:0/1251242724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4344101920 0x7f4344108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.233+0000 7f4348b4c640 1 --2- 172.21.15.112:0/1251242724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 0x7f43441013e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.233+0000 7f4348b4c640 1 --2- 172.21.15.112:0/1251242724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440ffe10 0x7f4344100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.233+0000 7f4348b4c640 1 -- 172.21.15.112:0/1251242724 >> 172.21.15.112:0/1251242724 conn(0x7f43440fb380 msgr2=0x7f43440fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 -- 172.21.15.112:0/1251242724 shutdown_connections 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 -- 172.21.15.112:0/1251242724 wait complete. 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 Processor -- start 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 -- start start 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440ffe10 0x7f4344197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 0x7f43441980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4344101920 0x7f434419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f434410b390 con 0x7f4344100f60 2024-04-24T23:18:23.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7f434410b210 con 0x7f43440ffe10 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.234+0000 7f4348b4c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f434410b510 con 0x7f4344101920 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 0x7f43441980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 0x7f43441980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50984/0 (socket says 172.21.15.112:50984) 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 -- 172.21.15.112:0/120597985 learned_addr learned my addr 172.21.15.112:0/120597985 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f434259c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440ffe10 0x7f4344197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4342d9d640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4344101920 0x7f434419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 -- 172.21.15.112:0/120597985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4344101920 msgr2=0x7f434419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4344101920 0x7f434419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 -- 172.21.15.112:0/120597985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440ffe10 msgr2=0x7f4344197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440ffe10 0x7f4344197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 -- 172.21.15.112:0/120597985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434419f790 con 0x7f4344100f60 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f434259c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440ffe10 0x7f4344197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:23.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4342d9d640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4344101920 0x7f434419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:18:23.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4341d9b640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 0x7f43441980b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f433800a340 tx=0x7f43380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:23.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f43337fe640 1 -- 172.21.15.112:0/120597985 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433803f070 con 0x7f4344100f60 2024-04-24T23:18:23.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f43337fe640 1 -- 172.21.15.112:0/120597985 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7f4338032400 con 0x7f4344100f60 2024-04-24T23:18:23.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434419fa20 con 0x7f4344100f60 2024-04-24T23:18:23.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f43337fe640 1 -- 172.21.15.112:0/120597985 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338036040 con 0x7f4344100f60 2024-04-24T23:18:23.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.235+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434419ff10 con 0x7f4344100f60 2024-04-24T23:18:23.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.236+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43440689b0 con 0x7f4344100f60 2024-04-24T23:18:23.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.237+0000 7f43337fe640 1 -- 172.21.15.112:0/120597985 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4338042070 con 0x7f4344100f60 2024-04-24T23:18:23.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.238+0000 7f43337fe640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f432c0a2b90 0x7f432c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:23.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.238+0000 7f434259c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f432c0a2b90 0x7f432c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:23.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.238+0000 7f43337fe640 1 -- 172.21.15.112:0/120597985 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9113+0+0 (secure 0 0 0) 0x7f43380efd60 con 0x7f4344100f60 2024-04-24T23:18:23.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.239+0000 7f434259c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f432c0a2b90 0x7f432c0a5050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f433400baa0 tx=0x7f4334005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:23.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.243+0000 7f43337fe640 1 -- 172.21.15.112:0/120597985 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f43380bc950 con 0x7f4344100f60 2024-04-24T23:18:23.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:23.420+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4344062340 con 0x7f4344100f60 2024-04-24T23:18:23.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:23 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2042012486' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:23.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:23 smithi187 ceph-mon[33292]: pgmap v473: 129 pgs: 129 active+clean; 109 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-24T23:18:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:23 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2042012486' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:23.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:23 smithi112 ceph-mon[29186]: pgmap v473: 129 pgs: 129 active+clean; 109 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-24T23:18:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:23 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2042012486' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:23.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:23 smithi150 ceph-mon[32247]: pgmap v473: 129 pgs: 129 active+clean; 109 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-24T23:18:24.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.120+0000 7f43337fe640 1 -- 172.21.15.112:0/120597985 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f43380bc250 con 0x7f4344100f60 2024-04-24T23:18:24.234 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-04-24T23:18:24.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.124+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f432c0a2b90 msgr2=0x7f432c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:24.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.124+0000 7f4348b4c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f432c0a2b90 0x7f432c0a5050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f433400baa0 tx=0x7f4334005d90 comp rx=0 tx=0).stop 2024-04-24T23:18:24.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.124+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 msgr2=0x7f43441980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:24.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.124+0000 7f4348b4c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 0x7f43441980b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f433800a340 tx=0x7f43380045e0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.124+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 shutdown_connections 2024-04-24T23:18:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.125+0000 7f4348b4c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7f432c0a2b90 0x7f432c0a5050 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.125+0000 7f4348b4c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4344101920 0x7f434419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.125+0000 7f4348b4c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4344100f60 0x7f43441980b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.125+0000 7f4348b4c640 1 --2- 172.21.15.112:0/120597985 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43440ffe10 0x7f4344197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.125+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 >> 172.21.15.112:0/120597985 conn(0x7f43440fb380 msgr2=0x7f4344104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.125+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 shutdown_connections 2024-04-24T23:18:24.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.125+0000 7f4348b4c640 1 -- 172.21.15.112:0/120597985 wait complete. 2024-04-24T23:18:24.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-24T23:18:24.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 -- 172.21.15.112:0/3372521056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a00ff940 msgr2=0x7ff4a00ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:24.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 --2- 172.21.15.112:0/3372521056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a00ff940 0x7ff4a00ffd60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff49c009900 tx=0x7ff49c02a400 comp rx=0 tx=0).stop 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 -- 172.21.15.112:0/3372521056 shutdown_connections 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 --2- 172.21.15.112:0/3372521056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4a0103370 0x7ff4a0109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 --2- 172.21.15.112:0/3372521056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01029b0 0x7ff4a0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 --2- 172.21.15.112:0/3372521056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a00ff940 0x7ff4a00ffd60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 -- 172.21.15.112:0/3372521056 >> 172.21.15.112:0/3372521056 conn(0x7ff4a00fb270 msgr2=0x7ff4a00fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 -- 172.21.15.112:0/3372521056 shutdown_connections 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.349+0000 7ff4a739e640 1 -- 172.21.15.112:0/3372521056 wait complete. 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a739e640 1 Processor -- start 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a739e640 1 -- start start 2024-04-24T23:18:24.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a739e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4a00ff940 0x7ff4a0199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a739e640 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01029b0 0x7ff4a019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a739e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a0103370 0x7ff4a01a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a739e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a010d450 con 0x7ff4a0103370 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a739e640 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a010d2d0 con 0x7ff4a01029b0 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a739e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a010d5d0 con 0x7ff4a00ff940 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a6b9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a0103370 0x7ff4a01a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a6b9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a0103370 0x7ff4a01a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51008/0 (socket says 172.21.15.112:51008) 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a639c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4a00ff940 0x7ff4a0199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.350+0000 7ff4a6b9d640 1 -- 172.21.15.112:0/131318267 learned_addr learned my addr 172.21.15.112:0/131318267 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-24T23:18:24.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a5b9b640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01029b0 0x7ff4a019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a6b9d640 1 -- 172.21.15.112:0/131318267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4a00ff940 msgr2=0x7ff4a0199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a6b9d640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4a00ff940 0x7ff4a0199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a6b9d640 1 -- 172.21.15.112:0/131318267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01029b0 msgr2=0x7ff4a019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a6b9d640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01029b0 0x7ff4a019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a6b9d640 1 -- 172.21.15.112:0/131318267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a01a18f0 con 0x7ff4a0103370 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a6b9d640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a0103370 0x7ff4a01a1250 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff49800eaa0 tx=0x7ff49800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4977fe640 1 -- 172.21.15.112:0/131318267 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49800cdb0 con 0x7ff4a0103370 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a01a1be0 con 0x7ff4a0103370 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4977fe640 1 -- 172.21.15.112:0/131318267 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(28 keys) ==== 1174+0+0 (secure 0 0 0) 0x7ff498004b20 con 0x7ff4a0103370 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4977fe640 1 -- 172.21.15.112:0/131318267 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498011760 con 0x7ff4a0103370 2024-04-24T23:18:24.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.351+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a01a2130 con 0x7ff4a0103370 2024-04-24T23:18:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.352+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a01a2360 con 0x7ff4a0103370 2024-04-24T23:18:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.353+0000 7ff4977fe640 1 -- 172.21.15.112:0/131318267 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff498011900 con 0x7ff4a0103370 2024-04-24T23:18:24.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.353+0000 7ff4977fe640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff48c0a2b90 0x7ff48c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:24.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.354+0000 7ff4a639c640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff48c0a2b90 0x7ff48c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:24.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.354+0000 7ff4977fe640 1 -- 172.21.15.112:0/131318267 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8496+0+0 (secure 0 0 0) 0x7ff4980cf390 con 0x7ff4a0103370 2024-04-24T23:18:24.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.354+0000 7ff4a639c640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff48c0a2b90 0x7ff48c0a5050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff4a019ab80 tx=0x7ff49c031640 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:24.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.358+0000 7ff4977fe640 1 -- 172.21.15.112:0/131318267 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff498016030 con 0x7ff4a0103370 2024-04-24T23:18:24.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:24 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/2042012486' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:18:24.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:24 smithi187 ceph-mon[33292]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:18:24.487 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:24 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/120597985' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:24.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:24 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/2042012486' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:18:24.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:24 smithi112 ceph-mon[29186]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:18:24.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:24 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/120597985' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:24.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:24.547+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff4a01a2530 con 0x7ff4a0103370 2024-04-24T23:18:24.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:24 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/2042012486' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:18:24.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:24 smithi150 ceph-mon[32247]: osdmap e96: 12 total, 12 up, 12 in 2024-04-24T23:18:24.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:24 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/120597985' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:25.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.126+0000 7ff4977fe640 1 -- 172.21.15.112:0/131318267 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) ==== 178+0+0 (secure 0 0 0) 0x7ff49809c1f0 con 0x7ff4a0103370 2024-04-24T23:18:25.128 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data_ec' removed 2024-04-24T23:18:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff48c0a2b90 msgr2=0x7ff48c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff48c0a2b90 0x7ff48c0a5050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff4a019ab80 tx=0x7ff49c031640 comp rx=0 tx=0).stop 2024-04-24T23:18:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a0103370 msgr2=0x7ff4a01a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a0103370 0x7ff4a01a1250 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff49800eaa0 tx=0x7ff49800ef60 comp rx=0 tx=0).stop 2024-04-24T23:18:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 shutdown_connections 2024-04-24T23:18:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:6800/2191305617,v1:172.21.15.112:6801/2191305617] conn(0x7ff48c0a2b90 0x7ff48c0a5050 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:25.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4a0103370 0x7ff4a01a1250 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:25.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a01029b0 0x7ff4a019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:25.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 --2- 172.21.15.112:0/131318267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4a00ff940 0x7ff4a0199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:25.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 >> 172.21.15.112:0/131318267 conn(0x7ff4a00fb270 msgr2=0x7ff4a01007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:25.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 shutdown_connections 2024-04-24T23:18:25.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-24T23:18:25.130+0000 7ff4a739e640 1 -- 172.21.15.112:0/131318267 wait complete. 2024-04-24T23:18:25.151 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:18:25.216 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:18:25.249 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:18:25.281 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T23:18:25.281 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:18:25.297 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:18:25.314 DEBUG:teuthology.orchestra.run.smithi187:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:18:25.338 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T23:18:25.338 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-24T23:18:25.339 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a 2024-04-24T23:18:25.380 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:25 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/120597985' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:18:25.380 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:25 smithi112 ceph-mon[29186]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:18:25.380 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:25 smithi112 ceph-mon[29186]: from='client.? 172.21.15.112:0/131318267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:25.380 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:25 smithi112 ceph-mon[29186]: pgmap v476: 33 pgs: 33 active+clean; 577 KiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:18:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:25 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/120597985' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:18:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:25 smithi187 ceph-mon[33292]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:18:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:25 smithi187 ceph-mon[33292]: from='client.? 172.21.15.112:0/131318267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:25.486 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:25 smithi187 ceph-mon[33292]: pgmap v476: 33 pgs: 33 active+clean; 577 KiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:18:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:25 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/120597985' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-24T23:18:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:25 smithi150 ceph-mon[32247]: osdmap e97: 12 total, 12 up, 12 in 2024-04-24T23:18:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:25 smithi150 ceph-mon[32247]: from='client.? 172.21.15.112:0/131318267' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-24T23:18:25.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:25 smithi150 ceph-mon[32247]: pgmap v476: 33 pgs: 33 active+clean; 577 KiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-24T23:18:25.696 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:25 smithi112 systemd[1]: Stopping Ceph mon.a for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:26.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a[29182]: 2024-04-24T23:18:25.694+0000 7fd109959700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:26.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a[29182]: 2024-04-24T23:18:25.694+0000 7fd109959700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T23:18:26.029 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:25 smithi112 podman[90699]: 2024-04-24 23:18:25.746094637 +0000 UTC m=+0.191916755 container died d2f995aeef1b83922afd350e42bb086710d07ca86cb6101e3309ad300d098f3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:18:26.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:26 smithi112 podman[90699]: 2024-04-24 23:18:26.270538521 +0000 UTC m=+0.716360638 container cleanup d2f995aeef1b83922afd350e42bb086710d07ca86cb6101e3309ad300d098f3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:18:26.286 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:26 smithi112 bash[90699]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a 2024-04-24T23:18:26.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a.service' 2024-04-24T23:18:26.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:26 smithi112 podman[90713]: 2024-04-24 23:18:26.497191393 +0000 UTC m=+0.744689593 container remove d2f995aeef1b83922afd350e42bb086710d07ca86cb6101e3309ad300d098f3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T23:18:26.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:26 smithi112 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-24T23:18:26.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:26 smithi112 systemd[1]: Stopped Ceph mon.a for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:18:26.779 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 24 23:18:26 smithi112 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.a.service: Consumed 14.853s CPU time. 2024-04-24T23:18:26.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:26.827 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-24T23:18:26.827 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-24T23:18:26.827 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.b 2024-04-24T23:18:27.142 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:26 smithi150 systemd[1]: Stopping Ceph mon.b for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:27.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:27 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-b[32243]: 2024-04-24T23:18:27.140+0000 7f8670ea5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:27 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-b[32243]: 2024-04-24T23:18:27.140+0000 7f8670ea5700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:18:27.569 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:27 smithi150 podman[65699]: 2024-04-24 23:18:27.294284822 +0000 UTC m=+0.276198696 container died 80ff5efbf2bea7bd9bcd985d228d9e4a4cb7e57611d929a8621cc7fb4f9c4b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:18:27.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:27 smithi150 podman[65699]: 2024-04-24 23:18:27.7251743 +0000 UTC m=+0.707088175 container cleanup 80ff5efbf2bea7bd9bcd985d228d9e4a4cb7e57611d929a8621cc7fb4f9c4b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:18:27.961 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:27 smithi150 bash[65699]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-b 2024-04-24T23:18:28.193 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.b.service' 2024-04-24T23:18:28.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:27 smithi150 podman[65713]: 2024-04-24 23:18:27.960209511 +0000 UTC m=+0.666255281 container remove 80ff5efbf2bea7bd9bcd985d228d9e4a4cb7e57611d929a8621cc7fb4f9c4b13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T23:18:28.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:28 smithi150 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.b.service: Deactivated successfully. 2024-04-24T23:18:28.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:28 smithi150 systemd[1]: Stopped Ceph mon.b for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:18:28.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Apr 24 23:18:28 smithi150 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.b.service: Consumed 8.926s CPU time. 2024-04-24T23:18:28.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:28.236 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-24T23:18:28.236 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-24T23:18:28.236 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.c 2024-04-24T23:18:28.545 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:28 smithi187 systemd[1]: Stopping Ceph mon.c for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:28.861 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:28 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-c[33288]: 2024-04-24T23:18:28.543+0000 7f611d789700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:28.862 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:28 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-c[33288]: 2024-04-24T23:18:28.543+0000 7f611d789700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:18:28.862 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:28 smithi187 podman[67130]: 2024-04-24 23:18:28.553978934 +0000 UTC m=+0.121804254 container died 0dfefceef0c15161601ca931bea3dcde40acffa5ea3b53c5b9d2671df1c02b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-c, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:18:29.172 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:28 smithi187 podman[67130]: 2024-04-24 23:18:28.960927442 +0000 UTC m=+0.528752765 container cleanup 0dfefceef0c15161601ca931bea3dcde40acffa5ea3b53c5b9d2671df1c02b2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-24T23:18:29.172 INFO:journalctl@ceph.mon.c.smithi187.stdout:Apr 24 23:18:28 smithi187 bash[67130]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mon-c 2024-04-24T23:18:29.382 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mon.c.service' 2024-04-24T23:18:29.445 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:29.445 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-24T23:18:29.446 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-24T23:18:29.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.x 2024-04-24T23:18:29.779 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:18:29 smithi112 systemd[1]: Stopping Ceph mgr.x for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:30.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:18:29 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:18:29.933+0000 7f63a2b6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:30.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:18:29 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x[29418]: 2024-04-24T23:18:29.933+0000 7f63a2b6d700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-24T23:18:30.271 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:18:29 smithi112 podman[90820]: 2024-04-24 23:18:29.965145998 +0000 UTC m=+0.330146647 container died ba5e22e86d481152f58933821f2fce9dc93b47d7977a77bd22fc18d4ee4a8af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:18:30.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:18:30 smithi112 podman[90820]: 2024-04-24 23:18:30.470489443 +0000 UTC m=+0.835490080 container cleanup ba5e22e86d481152f58933821f2fce9dc93b47d7977a77bd22fc18d4ee4a8af4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-04-24T23:18:30.529 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 24 23:18:30 smithi112 bash[90820]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-x 2024-04-24T23:18:30.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-24T23:18:30.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:30.890 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-24T23:18:30.890 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-24T23:18:30.890 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.y 2024-04-24T23:18:31.227 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:18:30 smithi150 systemd[1]: Stopping Ceph mgr.y for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:31.542 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:18:31 smithi150 podman[65826]: 2024-04-24 23:18:31.226217834 +0000 UTC m=+0.143494125 container died 17588345cc0cddd8620333cf9bcb2799c775136f58f69e797843148362945d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T23:18:31.818 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:18:31 smithi150 podman[65826]: 2024-04-24 23:18:31.639725158 +0000 UTC m=+0.557001448 container cleanup 17588345cc0cddd8620333cf9bcb2799c775136f58f69e797843148362945d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-24T23:18:31.818 INFO:journalctl@ceph.mgr.y.smithi150.stdout:Apr 24 23:18:31 smithi150 bash[65826]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-y 2024-04-24T23:18:32.014 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-24T23:18:32.047 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:32.047 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-24T23:18:32.047 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-24T23:18:32.047 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.z 2024-04-24T23:18:32.396 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:18:32 smithi187 systemd[1]: Stopping Ceph mgr.z for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:32.660 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:18:32 smithi187 podman[67253]: 2024-04-24 23:18:32.395511828 +0000 UTC m=+0.160627304 container died 1f38b294df7c69286e6e56930259385ecf0607796321cdfc1c750b908d341ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T23:18:32.986 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:18:32 smithi187 podman[67253]: 2024-04-24 23:18:32.772917671 +0000 UTC m=+0.538033156 container cleanup 1f38b294df7c69286e6e56930259385ecf0607796321cdfc1c750b908d341ed9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:18:32.986 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Apr 24 23:18:32 smithi187 bash[67253]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mgr-z 2024-04-24T23:18:33.341 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-24T23:18:33.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:33.374 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-24T23:18:33.374 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T23:18:33.374 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.0 2024-04-24T23:18:33.529 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:33 smithi112 systemd[1]: Stopping Ceph osd.0 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:34.029 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:33 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0[43928]: 2024-04-24T23:18:33.666+0000 7f76a350b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:34.029 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:33 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0[43928]: 2024-04-24T23:18:33.666+0000 7f76a350b700 -1 osd.0 98 *** Got signal Terminated *** 2024-04-24T23:18:34.029 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:33 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0[43928]: 2024-04-24T23:18:33.666+0000 7f76a350b700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:18:39.363 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:39 smithi112 podman[90944]: 2024-04-24 23:18:39.07036649 +0000 UTC m=+5.517592126 container died 20248e5a436a59b6bee045cecc8e8e14df81052d8b9e11e15843478e603136bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:18:39.779 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:39 smithi112 podman[90944]: 2024-04-24 23:18:39.569209409 +0000 UTC m=+6.016435056 container cleanup 20248e5a436a59b6bee045cecc8e8e14df81052d8b9e11e15843478e603136bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T23:18:39.779 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:39 smithi112 bash[90944]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0 2024-04-24T23:18:40.144 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:39 smithi112 podman[90961]: 2024-04-24 23:18:39.820891819 +0000 UTC m=+0.750883711 container remove 20248e5a436a59b6bee045cecc8e8e14df81052d8b9e11e15843478e603136bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-24T23:18:40.452 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:40 smithi112 podman[91053]: 2024-04-24 23:18:40.046185825 +0000 UTC m=+0.020326695 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:18:40.452 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:40 smithi112 podman[91053]: 2024-04-24 23:18:40.189032235 +0000 UTC m=+0.163173092 container create 94ce1272fc1cd46d5307aa475719f0c935ea53da9cb3b8e17a002d8136e59676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:18:40.452 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:40 smithi112 podman[91053]: 2024-04-24 23:18:40.450182696 +0000 UTC m=+0.424323555 container init 94ce1272fc1cd46d5307aa475719f0c935ea53da9cb3b8e17a002d8136e59676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:18:40.739 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:40 smithi112 podman[91053]: 2024-04-24 23:18:40.454047809 +0000 UTC m=+0.428188675 container start 94ce1272fc1cd46d5307aa475719f0c935ea53da9cb3b8e17a002d8136e59676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T23:18:40.739 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:40 smithi112 podman[91053]: 2024-04-24 23:18:40.5418716 +0000 UTC m=+0.516012478 container attach 94ce1272fc1cd46d5307aa475719f0c935ea53da9cb3b8e17a002d8136e59676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:18:41.029 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 24 23:18:40 smithi112 podman[91053]: 2024-04-24 23:18:40.738681673 +0000 UTC m=+0.712822531 container died 94ce1272fc1cd46d5307aa475719f0c935ea53da9cb3b8e17a002d8136e59676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T23:18:41.431 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T23:18:41.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:41.471 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T23:18:41.471 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T23:18:41.471 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.1 2024-04-24T23:18:41.717 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:41 smithi150 systemd[1]: Stopping Ceph osd.1 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:42.068 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:41 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1[36843]: 2024-04-24T23:18:41.801+0000 7f8dd30b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:42.068 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:41 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1[36843]: 2024-04-24T23:18:41.801+0000 7f8dd30b8700 -1 osd.1 98 *** Got signal Terminated *** 2024-04-24T23:18:42.068 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:41 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1[36843]: 2024-04-24T23:18:41.801+0000 7f8dd30b8700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:18:47.431 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:47 smithi150 podman[65950]: 2024-04-24 23:18:47.135500541 +0000 UTC m=+5.474547893 container died d1e55a960d0333f6f5e434dac7c020dca3d0e6728bd47a5c18019eb1901d43e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T23:18:47.773 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:47 smithi150 podman[65950]: 2024-04-24 23:18:47.553037368 +0000 UTC m=+5.892084710 container cleanup d1e55a960d0333f6f5e434dac7c020dca3d0e6728bd47a5c18019eb1901d43e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T23:18:47.773 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:47 smithi150 bash[65950]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1 2024-04-24T23:18:48.068 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:47 smithi150 podman[65979]: 2024-04-24 23:18:47.771587657 +0000 UTC m=+0.638044103 container remove d1e55a960d0333f6f5e434dac7c020dca3d0e6728bd47a5c18019eb1901d43e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T23:18:48.510 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:48 smithi150 podman[66074]: 2024-04-24 23:18:48.122253959 +0000 UTC m=+0.018529248 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:18:48.510 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:48 smithi150 podman[66074]: 2024-04-24 23:18:48.239110009 +0000 UTC m=+0.135385298 container create 36aaa79e86d80c5da7aa4e4a1b696845ff39a09eae72fbc3922f15e5ecdc0482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:18:48.510 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:48 smithi150 podman[66074]: 2024-04-24 23:18:48.508803469 +0000 UTC m=+0.405078758 container init 36aaa79e86d80c5da7aa4e4a1b696845ff39a09eae72fbc3922f15e5ecdc0482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T23:18:48.803 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:48 smithi150 podman[66074]: 2024-04-24 23:18:48.51213228 +0000 UTC m=+0.408407570 container start 36aaa79e86d80c5da7aa4e4a1b696845ff39a09eae72fbc3922f15e5ecdc0482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:18:48.804 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:48 smithi150 podman[66074]: 2024-04-24 23:18:48.617221001 +0000 UTC m=+0.513496300 container attach 36aaa79e86d80c5da7aa4e4a1b696845ff39a09eae72fbc3922f15e5ecdc0482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:18:49.068 INFO:journalctl@ceph.osd.1.smithi150.stdout:Apr 24 23:18:48 smithi150 podman[66074]: 2024-04-24 23:18:48.80339038 +0000 UTC m=+0.699665678 container died 36aaa79e86d80c5da7aa4e4a1b696845ff39a09eae72fbc3922f15e5ecdc0482 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:18:49.499 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T23:18:49.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:49.544 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T23:18:49.544 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T23:18:49.544 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.2 2024-04-24T23:18:49.849 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:49 smithi187 systemd[1]: Stopping Ceph osd.2 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:50.236 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2[38563]: 2024-04-24T23:18:49.847+0000 7f46cefa1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:50.236 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2[38563]: 2024-04-24T23:18:49.847+0000 7f46cefa1700 -1 osd.2 98 *** Got signal Terminated *** 2024-04-24T23:18:50.236 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2[38563]: 2024-04-24T23:18:49.847+0000 7f46cefa1700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:18:55.667 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:55 smithi187 podman[67390]: 2024-04-24 23:18:55.370015474 +0000 UTC m=+5.636744185 container died 457f4cc6daf65346b75d136f99692f1428e9c14c958cafb2009740ecb2801c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T23:18:55.986 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:55 smithi187 podman[67390]: 2024-04-24 23:18:55.764873526 +0000 UTC m=+6.031602237 container cleanup 457f4cc6daf65346b75d136f99692f1428e9c14c958cafb2009740ecb2801c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T23:18:55.986 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:55 smithi187 bash[67390]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2 2024-04-24T23:18:56.387 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:56 smithi187 podman[67406]: 2024-04-24 23:18:56.033126567 +0000 UTC m=+0.658576577 container remove 457f4cc6daf65346b75d136f99692f1428e9c14c958cafb2009740ecb2801c83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:18:56.647 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:56 smithi187 podman[67495]: 2024-04-24 23:18:56.28971888 +0000 UTC m=+0.018306285 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:18:56.647 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:56 smithi187 podman[67495]: 2024-04-24 23:18:56.417718574 +0000 UTC m=+0.146305997 container create f5afedcce2c9cfa2feffbc7fc53033f0bb3f3f9c591e2bf3a8ffe9ef53233807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T23:18:56.945 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:56 smithi187 podman[67495]: 2024-04-24 23:18:56.645720308 +0000 UTC m=+0.374307686 container init f5afedcce2c9cfa2feffbc7fc53033f0bb3f3f9c591e2bf3a8ffe9ef53233807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T23:18:56.945 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:56 smithi187 podman[67495]: 2024-04-24 23:18:56.649494565 +0000 UTC m=+0.378081943 container start f5afedcce2c9cfa2feffbc7fc53033f0bb3f3f9c591e2bf3a8ffe9ef53233807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:18:56.945 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:56 smithi187 podman[67495]: 2024-04-24 23:18:56.745899965 +0000 UTC m=+0.474487351 container attach f5afedcce2c9cfa2feffbc7fc53033f0bb3f3f9c591e2bf3a8ffe9ef53233807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:18:57.236 INFO:journalctl@ceph.osd.2.smithi187.stdout:Apr 24 23:18:56 smithi187 podman[67495]: 2024-04-24 23:18:56.944506737 +0000 UTC m=+0.673094125 container died f5afedcce2c9cfa2feffbc7fc53033f0bb3f3f9c591e2bf3a8ffe9ef53233807 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T23:18:57.639 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T23:18:57.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:18:57.684 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T23:18:57.684 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T23:18:57.684 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.3 2024-04-24T23:18:57.996 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:18:57 smithi112 systemd[1]: Stopping Ceph osd.3 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:58.268 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:18:58 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3[50879]: 2024-04-24T23:18:58.082+0000 7f91cb487700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:58.268 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:18:58 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3[50879]: 2024-04-24T23:18:58.082+0000 7f91cb487700 -1 osd.3 98 *** Got signal Terminated *** 2024-04-24T23:18:58.268 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:18:58 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3[50879]: 2024-04-24T23:18:58.082+0000 7f91cb487700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:00.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:00 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:00.198+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:00.986 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:00 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:00.590+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:01.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:01.181+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:01.780 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:01 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3[50879]: 2024-04-24T23:19:01.616+0000 7f91c7a7b700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:38.045318+0000 front 2024-04-24T23:18:38.045343+0000 (oldest deadline 2024-04-24T23:19:01.545472+0000) 2024-04-24T23:19:01.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:01 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:01.412+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:01.986 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:01.554+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:02.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:02.189+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:02.779 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:02 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3[50879]: 2024-04-24T23:19:02.598+0000 7f91c7a7b700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:38.045318+0000 front 2024-04-24T23:18:38.045343+0000 (oldest deadline 2024-04-24T23:19:01.545472+0000) 2024-04-24T23:19:02.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:02 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:02.409+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:02.986 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:02.584+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:03.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:03.201+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:03.769 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:03 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:03.459+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:03.769 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:03 smithi112 podman[91163]: 2024-04-24 23:19:03.437450286 +0000 UTC m=+5.567968492 container died f5ec225177bfbc357dbdaaf0f2ead20b69116a3609fa4b47ca693ae0330ecca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T23:19:03.986 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:03.545+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:04.029 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:03 smithi112 podman[91163]: 2024-04-24 23:19:03.958418178 +0000 UTC m=+6.088936383 container cleanup f5ec225177bfbc357dbdaaf0f2ead20b69116a3609fa4b47ca693ae0330ecca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T23:19:04.029 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:03 smithi112 bash[91163]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3 2024-04-24T23:19:04.477 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:04 smithi112 podman[91192]: 2024-04-24 23:19:04.210119719 +0000 UTC m=+0.775718335 container remove f5ec225177bfbc357dbdaaf0f2ead20b69116a3609fa4b47ca693ae0330ecca9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T23:19:04.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:04.232+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:04.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:04 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:04.475+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:04.779 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:04 smithi112 podman[91287]: 2024-04-24 23:19:04.535106162 +0000 UTC m=+0.018332222 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:04.779 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:04 smithi112 podman[91287]: 2024-04-24 23:19:04.661358057 +0000 UTC m=+0.144584143 container create a0dfda653bf7edfa92de9252ff05d3a704e0265dc04dfe4a1647dae73e5372f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:19:04.986 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:04.511+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:05.221 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:04 smithi112 podman[91287]: 2024-04-24 23:19:04.930501133 +0000 UTC m=+0.413727176 container init a0dfda653bf7edfa92de9252ff05d3a704e0265dc04dfe4a1647dae73e5372f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T23:19:05.221 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:04 smithi112 podman[91287]: 2024-04-24 23:19:04.933739715 +0000 UTC m=+0.416965766 container start a0dfda653bf7edfa92de9252ff05d3a704e0265dc04dfe4a1647dae73e5372f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T23:19:05.221 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:05 smithi112 podman[91287]: 2024-04-24 23:19:05.02229096 +0000 UTC m=+0.505517006 container attach a0dfda653bf7edfa92de9252ff05d3a704e0265dc04dfe4a1647dae73e5372f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T23:19:05.221 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 24 23:19:05 smithi112 podman[91287]: 2024-04-24 23:19:05.220006557 +0000 UTC m=+0.703232605 container died a0dfda653bf7edfa92de9252ff05d3a704e0265dc04dfe4a1647dae73e5372f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T23:19:05.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:05 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:05.437+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:05.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:05.470+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:05.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:05.263+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:05.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T23:19:06.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:19:06.025 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T23:19:06.025 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T23:19:06.025 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.4 2024-04-24T23:19:06.318 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:06 smithi150 systemd[1]: Stopping Ceph osd.4 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:06.496 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:06.237+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:06.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:06 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:06.465+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:06.818 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:06 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:19:06.365+0000 7ffb93d63700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:06.818 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:06 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:19:06.365+0000 7ffb93d63700 -1 osd.4 98 *** Got signal Terminated *** 2024-04-24T23:19:06.818 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:06 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:19:06.365+0000 7ffb93d63700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:06.986 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:06.494+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:07.527 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:07 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:07.247+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:07.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:07 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:07.424+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:07.986 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:07 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:07.525+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:08.198 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:07 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:07.909+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:08.568 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:08 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:19:08.196+0000 7ffb90357700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:41.540862+0000 front 2024-04-24T23:18:41.540932+0000 (oldest deadline 2024-04-24T23:19:07.341350+0000) 2024-04-24T23:19:08.701 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:08 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:08.434+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:08.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:08 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:08.507+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:08.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:08 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:08.292+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:09.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:08 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:08.954+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:09.318 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:09 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:19:09.162+0000 7ffb90357700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:41.540862+0000 front 2024-04-24T23:18:41.540932+0000 (oldest deadline 2024-04-24T23:19:07.341350+0000) 2024-04-24T23:19:09.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:09 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:09.474+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:09.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:09 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:09.517+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:09.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:09 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:09.285+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:10.029 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:09 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:09.637+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:10.318 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:10 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:19:10.113+0000 7ffb90357700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:41.540862+0000 front 2024-04-24T23:18:41.540932+0000 (oldest deadline 2024-04-24T23:19:07.341350+0000) 2024-04-24T23:19:10.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:09 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:09.978+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:10.489 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:10 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:10.167+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:10.489 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:10 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:10.307+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:10.657 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:10 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:10.322+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:10.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:10 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:10.451+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:10.780 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:10 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:10.659+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:10.986 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:10 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:10.525+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:11.282 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:11 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4[43614]: 2024-04-24T23:19:11.065+0000 7ffb90357700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:41.540862+0000 front 2024-04-24T23:18:41.540932+0000 (oldest deadline 2024-04-24T23:19:07.341350+0000) 2024-04-24T23:19:11.283 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:10 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:10.951+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:11.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:11 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:11.320+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:11.486 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:11 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:11.165+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:11.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:11 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:11.281+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:11.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:11 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:11.483+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:11.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:11 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:11.673+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:11.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:11 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:11.468+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:12.035 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:11 smithi150 podman[66178]: 2024-04-24 23:19:11.731341772 +0000 UTC m=+5.506397384 container died e50706f92cf0dcf346f63a16bb92a69f6ee960a9452ca1e1ce16d95912ad4341 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-24T23:19:12.035 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:11 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:11.924+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:12.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:12 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:12.257+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:12.318 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:12 smithi150 podman[66178]: 2024-04-24 23:19:12.133036586 +0000 UTC m=+5.908092187 container cleanup e50706f92cf0dcf346f63a16bb92a69f6ee960a9452ca1e1ce16d95912ad4341 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:12.318 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:12 smithi150 bash[66178]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4 2024-04-24T23:19:12.460 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:12 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:12.322+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:12.460 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:12 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:12.205+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:12.702 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:12 smithi150 podman[66208]: 2024-04-24 23:19:12.401623054 +0000 UTC m=+0.671121866 container remove e50706f92cf0dcf346f63a16bb92a69f6ee960a9452ca1e1ce16d95912ad4341 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T23:19:12.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:12 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:12.458+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:12.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:12 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:12.458+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.004826+0000 front 2024-04-24T23:18:46.005006+0000 (oldest deadline 2024-04-24T23:19:11.904841+0000) 2024-04-24T23:19:12.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:12 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:12.648+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:12.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:12 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:12.434+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:12.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:12 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:12.434+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:12.956 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:12 smithi150 podman[66301]: 2024-04-24 23:19:12.604865609 +0000 UTC m=+0.018355387 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:12.956 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:12 smithi150 podman[66301]: 2024-04-24 23:19:12.702338073 +0000 UTC m=+0.115827851 container create e1c291210e8085115737029edd1b7f61a67fed354467574f341cd1e7d3746487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T23:19:12.956 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:12 smithi150 podman[66301]: 2024-04-24 23:19:12.954981112 +0000 UTC m=+0.368470899 container init e1c291210e8085115737029edd1b7f61a67fed354467574f341cd1e7d3746487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:19:13.220 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:12 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:12.971+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:13.221 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:12 smithi150 podman[66301]: 2024-04-24 23:19:12.95832535 +0000 UTC m=+0.371815129 container start e1c291210e8085115737029edd1b7f61a67fed354467574f341cd1e7d3746487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:19:13.221 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:13 smithi150 podman[66301]: 2024-04-24 23:19:13.096937858 +0000 UTC m=+0.510427646 container attach e1c291210e8085115737029edd1b7f61a67fed354467574f341cd1e7d3746487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-24T23:19:13.481 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:13 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:13.187+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:13.481 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:13 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:13.371+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:13.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:13 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:13.218+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:13.568 INFO:journalctl@ceph.osd.4.smithi150.stdout:Apr 24 23:19:13 smithi150 podman[66301]: 2024-04-24 23:19:13.24478444 +0000 UTC m=+0.658274211 container died e1c291210e8085115737029edd1b7f61a67fed354467574f341cd1e7d3746487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:19:13.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:13 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:13.479+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:13.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:13 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:13.479+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.004826+0000 front 2024-04-24T23:18:46.005006+0000 (oldest deadline 2024-04-24T23:19:11.904841+0000) 2024-04-24T23:19:13.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:13 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:13.644+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:13.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:13 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:13.474+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:13.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:13 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:13.474+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:13.936 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T23:19:13.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:19:13.973 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T23:19:13.974 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T23:19:13.974 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.5 2024-04-24T23:19:14.232 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:13 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:13.948+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:14.288 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:14 smithi187 systemd[1]: Stopping Ceph osd.5 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:14.289 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:14 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:14.237+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:14.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:14 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:14.231+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:14.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:14 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:14.334+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:14.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:14 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:14.286+0000 7f6a6b707700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:14.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:14 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:14.286+0000 7f6a6b707700 -1 osd.5 98 *** Got signal Terminated *** 2024-04-24T23:19:14.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:14 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:14.286+0000 7f6a6b707700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:14.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:14 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:14.469+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:14.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:14 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:14.469+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.004826+0000 front 2024-04-24T23:18:46.005006+0000 (oldest deadline 2024-04-24T23:19:11.904841+0000) 2024-04-24T23:19:14.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:14 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:14.597+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:14.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:14 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:14.491+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:14.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:14 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:14.491+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:15.262 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:14 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:14.937+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:15.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:15 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:15.260+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:15.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:15 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:15.327+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:15.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:15 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:15.258+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:15.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:15 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:15.447+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:15.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:15 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:15.447+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.004826+0000 front 2024-04-24T23:18:46.005006+0000 (oldest deadline 2024-04-24T23:19:11.904841+0000) 2024-04-24T23:19:15.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:15 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:15.569+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:15.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:15 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:15.486+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:15.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:15 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:15.486+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:16.292 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:15 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:15.905+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:16.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:16 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:16.291+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:16.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:16 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:16.291+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:16.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:16 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:16.334+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:16.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:16 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:16.334+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:16.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:16 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:16.266+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:16.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:16 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:16.266+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:16.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:16 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:16.458+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:16.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:16 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:16.458+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.004826+0000 front 2024-04-24T23:18:46.005006+0000 (oldest deadline 2024-04-24T23:19:11.904841+0000) 2024-04-24T23:19:16.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:16 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:16.458+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.706102+0000 front 2024-04-24T23:18:54.706041+0000 (oldest deadline 2024-04-24T23:19:16.406149+0000) 2024-04-24T23:19:16.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:16 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:16.603+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:16.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:16 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:16.437+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:16.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:16 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:16.437+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:16.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:16 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:16.437+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:17.281 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:16 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:16.929+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:17.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:17 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:17.280+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:17.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:17 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:17.280+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:17.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:17 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:17.292+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:17.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:17 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:17.292+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:17.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:17 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:17.237+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:17.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:17 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:17.237+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:17.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:17 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:17.462+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:17.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:17 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:17.462+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.004826+0000 front 2024-04-24T23:18:46.005006+0000 (oldest deadline 2024-04-24T23:19:11.904841+0000) 2024-04-24T23:19:17.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:17 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:17.462+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.706102+0000 front 2024-04-24T23:18:54.706041+0000 (oldest deadline 2024-04-24T23:19:16.406149+0000) 2024-04-24T23:19:17.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:17 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:17.564+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:17.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:17 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:17.483+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:17.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:17 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:17.483+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:17.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:17 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:17.483+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:18.264 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:17 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:17.881+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:18.264 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:17 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:17.881+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:18.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:18 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:18.262+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:18.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:18 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:18.262+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:18.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:18 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:18.331+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:18.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:18 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:18.331+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:18.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:18 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:18.268+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:18.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:18 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:18.268+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:18.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:18 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:18.503+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:34.303422+0000 front 2024-04-24T23:18:34.303448+0000 (oldest deadline 2024-04-24T23:19:00.103947+0000) 2024-04-24T23:19:18.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:18 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:18.503+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.004826+0000 front 2024-04-24T23:18:46.005006+0000 (oldest deadline 2024-04-24T23:19:11.904841+0000) 2024-04-24T23:19:18.737 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:18 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5[45110]: 2024-04-24T23:19:18.503+0000 7f6a63cf3700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.706102+0000 front 2024-04-24T23:18:54.706041+0000 (oldest deadline 2024-04-24T23:19:16.406149+0000) 2024-04-24T23:19:18.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:18 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:18.600+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:18.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:18 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:18.487+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:18.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:18 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:18.487+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:18.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:18 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:18.487+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:19.225 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:18 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:18.926+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:19.225 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:18 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:18.926+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:19.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:19 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:19.224+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:19.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:19 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:19.224+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:19.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:19 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:19.287+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:19.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:19 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:19.287+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:19.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:19 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:19.304+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:19.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:19 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:19.304+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:19.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:19.584+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:19.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:19.584+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.128465+0000 front 2024-04-24T23:18:54.128429+0000 (oldest deadline 2024-04-24T23:19:18.828349+0000) 2024-04-24T23:19:19.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:19.489+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:19.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:19.489+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:19.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:19.489+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:20.157 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:19 smithi187 podman[67612]: 2024-04-24 23:19:19.833349825 +0000 UTC m=+5.670198338 container died 71fadb5b49cba9e567481c1b21f1a088d8d5c540586d1777c83284362812d8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:20.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:19 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:19.943+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:20.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:19 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:19.943+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:20.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:20.190+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:20.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:20.190+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:20.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:20.328+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:20.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:20.328+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:20.487 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:20 smithi187 podman[67612]: 2024-04-24 23:19:20.246195638 +0000 UTC m=+6.083044167 container cleanup 71fadb5b49cba9e567481c1b21f1a088d8d5c540586d1777c83284362812d8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-24T23:19:20.487 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:20 smithi187 bash[67612]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5 2024-04-24T23:19:20.487 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:20.302+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:20.487 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:20 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:20.302+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:20.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:20.565+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:20.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:20.565+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.128465+0000 front 2024-04-24T23:18:54.128429+0000 (oldest deadline 2024-04-24T23:19:18.828349+0000) 2024-04-24T23:19:20.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:20.454+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:20.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:20.454+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:20.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:20 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:20.454+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:20.905 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:20 smithi187 podman[67628]: 2024-04-24 23:19:20.489442855 +0000 UTC m=+0.657586227 container remove 71fadb5b49cba9e567481c1b21f1a088d8d5c540586d1777c83284362812d8a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T23:19:21.179 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:20.990+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:21.179 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:20 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:20.990+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:21.180 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:21 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:21.178+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:21.180 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:21 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:21.178+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:21.186 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:20 smithi187 podman[67722]: 2024-04-24 23:19:20.807113538 +0000 UTC m=+0.018709592 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:21.186 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:20 smithi187 podman[67722]: 2024-04-24 23:19:20.915590154 +0000 UTC m=+0.127186240 container create 88c84089459bf27180af6f9a4f51efca5fe5e369a2043d9cfd8aeecf3d4daaad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:19:21.186 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:21 smithi187 podman[67722]: 2024-04-24 23:19:21.185092798 +0000 UTC m=+0.396688846 container init 88c84089459bf27180af6f9a4f51efca5fe5e369a2043d9cfd8aeecf3d4daaad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:21.480 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:21 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:21.302+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:21.481 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:21 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:21.302+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:21.481 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:21 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:21.371+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:21.481 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:21 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:21.371+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:21.481 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:21 smithi187 podman[67722]: 2024-04-24 23:19:21.18817629 +0000 UTC m=+0.399772341 container start 88c84089459bf27180af6f9a4f51efca5fe5e369a2043d9cfd8aeecf3d4daaad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:19:21.481 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:21 smithi187 podman[67722]: 2024-04-24 23:19:21.285311363 +0000 UTC m=+0.496907418 container attach 88c84089459bf27180af6f9a4f51efca5fe5e369a2043d9cfd8aeecf3d4daaad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:19:21.736 INFO:journalctl@ceph.osd.5.smithi187.stdout:Apr 24 23:19:21 smithi187 podman[67722]: 2024-04-24 23:19:21.479588708 +0000 UTC m=+0.691184743 container died 88c84089459bf27180af6f9a4f51efca5fe5e369a2043d9cfd8aeecf3d4daaad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-04-24T23:19:21.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:21 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:21.591+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:21.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:21 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:21.591+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.128465+0000 front 2024-04-24T23:18:54.128429+0000 (oldest deadline 2024-04-24T23:19:18.828349+0000) 2024-04-24T23:19:21.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:21 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:21.444+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:21.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:21 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:21.444+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:21.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:21 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:21.444+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:22.095 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T23:19:22.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:19:22.133 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T23:19:22.133 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-24T23:19:22.133 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.6 2024-04-24T23:19:22.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:22 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:22.029+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:22.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:22 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:22.029+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:22.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:22 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:22.167+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:22.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:22 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:22.167+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:22.414 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:22 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:22.307+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:22.414 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:22 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:22.307+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:22.454 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:22 smithi112 systemd[1]: Stopping Ceph osd.6 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:22.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:22 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:22.413+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:22.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:22 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:22.413+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:22.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:22.452+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:22.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:22.452+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:22.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:22.452+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:22.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:22.454+0000 7fbd05f91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:22.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:22.454+0000 7fbd05f91700 -1 osd.6 98 *** Got signal Terminated *** 2024-04-24T23:19:22.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:22.454+0000 7fbd05f91700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:22.780 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:22.543+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:22.780 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:22.543+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.128465+0000 front 2024-04-24T23:18:54.128429+0000 (oldest deadline 2024-04-24T23:19:18.828349+0000) 2024-04-24T23:19:23.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:23 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:23.043+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:23.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:23 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:23.043+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:23.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:23 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:23.131+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:23.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:23 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:23.131+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:23.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:23 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:23.274+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:23.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:23 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:23.274+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:23.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:23 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:23.381+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:23.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:23 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:23.381+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:23.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:23 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:23.550+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:23.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:23 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:23.550+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.128465+0000 front 2024-04-24T23:18:54.128429+0000 (oldest deadline 2024-04-24T23:19:18.828349+0000) 2024-04-24T23:19:23.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:23 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:23.478+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:23.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:23 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:23.478+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:23.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:23 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:23.478+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:24.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:24 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:24.005+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:24.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:24 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:24.005+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:24.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:24 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:24.159+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:24.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:24 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:24.159+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:24.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:24 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:24.242+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:24.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:24 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:24.242+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:24.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:24 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:24.331+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:24.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:24 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:24.331+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:24.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:24.590+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:24.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:24.590+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.128465+0000 front 2024-04-24T23:18:54.128429+0000 (oldest deadline 2024-04-24T23:19:18.828349+0000) 2024-04-24T23:19:24.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:24.590+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.829888+0000 front 2024-04-24T23:19:02.829833+0000 (oldest deadline 2024-04-24T23:19:24.529880+0000) 2024-04-24T23:19:24.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:24.474+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:24.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:24.474+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:24.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:24 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:24.474+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:25.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:25 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:25.133+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:25.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:25 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:25.133+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:25.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:24 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:24.966+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:25.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:24 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:24.966+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:25.738 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:25 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:25.283+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:25.738 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:25 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:25.283+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:25.738 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:25 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:25.337+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:25.738 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:25 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:25.337+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:25.738 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:25 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:25.337+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:25.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:25.557+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:25.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:25.557+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.128465+0000 front 2024-04-24T23:18:54.128429+0000 (oldest deadline 2024-04-24T23:19:18.828349+0000) 2024-04-24T23:19:25.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:25.557+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.829888+0000 front 2024-04-24T23:19:02.829833+0000 (oldest deadline 2024-04-24T23:19:24.529880+0000) 2024-04-24T23:19:25.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:25.466+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:25.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:25.466+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:25.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:25 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:25.466+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:26.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:25 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:25.978+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:26.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:25 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:25.978+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:26.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:26 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:26.097+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:26.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:26 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:26.097+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:26.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:26 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:26.097+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:26.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:26 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:26.324+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:26.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:26 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:26.324+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:26.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:26 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:26.324+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:26.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:26 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:26.383+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:26.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:26 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:26.383+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:26.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:26 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:26.383+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:26.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:26 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:26.582+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:44.127235+0000 front 2024-04-24T23:18:44.127364+0000 (oldest deadline 2024-04-24T23:19:09.427237+0000) 2024-04-24T23:19:26.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:26 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:26.582+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:54.128465+0000 front 2024-04-24T23:18:54.128429+0000 (oldest deadline 2024-04-24T23:19:18.828349+0000) 2024-04-24T23:19:26.779 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:26 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6[57858]: 2024-04-24T23:19:26.582+0000 7fbd02585700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.829888+0000 front 2024-04-24T23:19:02.829833+0000 (oldest deadline 2024-04-24T23:19:24.529880+0000) 2024-04-24T23:19:26.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:26 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:26.465+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:26.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:26 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:26.465+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:26.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:26 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:26.465+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:27.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:27 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:27.010+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:27.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:27 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:27.010+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:27.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:27 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:27.107+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:27.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:27 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:27.107+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:27.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:27 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:27.107+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:27.693 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:27 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:27.432+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:27.693 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:27 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:27.432+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:27.693 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:27 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:27.432+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:27.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:27 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:27.340+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:27.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:27 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:27.340+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:27.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:27 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:27.340+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:27.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:27 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:27.424+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:27.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:27 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:27.424+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:27.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:27 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:27.424+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:28.240 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:27 smithi112 podman[91392]: 2024-04-24 23:19:27.98776097 +0000 UTC m=+5.658710158 container died 33e505153f534d571bcc2f2906ff2e11fa470a8c0462a3c35cd7879831474b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:19:28.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:28 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:28.033+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:28.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:28 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:28.033+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:28.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:28 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:28.033+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.882839+0000 front 2024-04-24T23:19:02.882739+0000 (oldest deadline 2024-04-24T23:19:27.582708+0000) 2024-04-24T23:19:28.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:28 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:28.112+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:28.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:28 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:28.112+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:28.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:28 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:28.112+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:28.529 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:28 smithi112 podman[91392]: 2024-04-24 23:19:28.413813249 +0000 UTC m=+6.084762430 container cleanup 33e505153f534d571bcc2f2906ff2e11fa470a8c0462a3c35cd7879831474b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:19:28.529 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:28 smithi112 bash[91392]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6 2024-04-24T23:19:28.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:28 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:28.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:28.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:28 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:28.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:28.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:28 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:28.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:28.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:28 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:28.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:28.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:28 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:28.367+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:28.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:28 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:28.367+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:28.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:28 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:28.367+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:28.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:28 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:28.398+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:28.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:28 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:28.398+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:28.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:28 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:28.398+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:28.909 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:28 smithi112 podman[91411]: 2024-04-24 23:19:28.615020731 +0000 UTC m=+0.631813712 container remove 33e505153f534d571bcc2f2906ff2e11fa470a8c0462a3c35cd7879831474b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:19:29.225 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:29 smithi112 podman[91505]: 2024-04-24 23:19:28.935359531 +0000 UTC m=+0.018660693 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:29.226 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:29 smithi112 podman[91505]: 2024-04-24 23:19:29.057837704 +0000 UTC m=+0.141138865 container create 07379f2e0ad4f2694499bdc94f034dc2b463dab986d525a811d0df364c6278d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:19:29.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:29 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:29.436+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:29.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:29 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:29.436+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:29.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:29 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:29.436+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:29.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:29 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:29.436+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:29.530 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:29 smithi112 podman[91505]: 2024-04-24 23:19:29.377569939 +0000 UTC m=+0.460871100 container init 07379f2e0ad4f2694499bdc94f034dc2b463dab986d525a811d0df364c6278d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-24T23:19:29.530 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:29 smithi112 podman[91505]: 2024-04-24 23:19:29.380850773 +0000 UTC m=+0.464151943 container start 07379f2e0ad4f2694499bdc94f034dc2b463dab986d525a811d0df364c6278d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:19:29.530 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:29 smithi112 podman[91505]: 2024-04-24 23:19:29.452367799 +0000 UTC m=+0.535668963 container attach 07379f2e0ad4f2694499bdc94f034dc2b463dab986d525a811d0df364c6278d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T23:19:29.568 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:29 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:29.066+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:29.568 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:29 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:29.066+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:29.568 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:29 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:29.066+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.882839+0000 front 2024-04-24T23:19:02.882739+0000 (oldest deadline 2024-04-24T23:19:27.582708+0000) 2024-04-24T23:19:29.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:29 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:29.107+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:29.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:29 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:29.107+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:29.569 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:29 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:29.107+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:29.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:29 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:29.340+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:29.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:29 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:29.340+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:29.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:29 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:29.340+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:29.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:29 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:29.389+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:29.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:29 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:29.389+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:29.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:29 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:29.389+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:29.956 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 24 23:19:29 smithi112 podman[91505]: 2024-04-24 23:19:29.669258783 +0000 UTC m=+0.752559949 container died 07379f2e0ad4f2694499bdc94f034dc2b463dab986d525a811d0df364c6278d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T23:19:30.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:30.048+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:30.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:30.048+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:30.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:30.048+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.882839+0000 front 2024-04-24T23:19:02.882739+0000 (oldest deadline 2024-04-24T23:19:27.582708+0000) 2024-04-24T23:19:30.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:30.115+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:30.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:30.115+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:30.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:30.115+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:30.395 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.6.service' 2024-04-24T23:19:30.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:19:30.435 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-24T23:19:30.435 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-24T23:19:30.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.7 2024-04-24T23:19:30.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:30 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:30.299+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:30.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:30 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:30.299+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:30.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:30 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:30.299+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:30.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:30 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:30.353+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:30.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:30 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:30.353+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:30.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:30 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:30.353+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:30.747 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:30 smithi150 systemd[1]: Stopping Ceph osd.7 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:30.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:30 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:30.426+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:30.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:30 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:30.426+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:30.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:30 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:30.426+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:30.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:30 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:30.426+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:31.059 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:30.744+0000 7f1fa5a54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:31.060 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:30.744+0000 7f1fa5a54700 -1 osd.7 98 *** Got signal Terminated *** 2024-04-24T23:19:31.060 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:30 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:30.744+0000 7f1fa5a54700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:31.060 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:31 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:31.057+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:31.060 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:31 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:31.057+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:31.060 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:31 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:31.057+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.882839+0000 front 2024-04-24T23:19:02.882739+0000 (oldest deadline 2024-04-24T23:19:27.582708+0000) 2024-04-24T23:19:31.127 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:31 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:31.123+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:31.128 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:31 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:31.123+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:31.128 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:31 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:31.123+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:31.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:31 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:31.319+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:31.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:31 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:31.319+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:31.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:31 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:31.319+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:31.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:31 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:31.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:31.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:31 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:31.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:31.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:31 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:31.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:31.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:31 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:31.451+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:31.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:31 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:31.451+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:31.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:31 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:31.451+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:31.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:31 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:31.451+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:32.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:32 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:32.135+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:32.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:32 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:32.135+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:32.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:32 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:32.135+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:32.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:32 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:32.052+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:32.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:32 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:32.052+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:32.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:32 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:32.052+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.882839+0000 front 2024-04-24T23:19:02.882739+0000 (oldest deadline 2024-04-24T23:19:27.582708+0000) 2024-04-24T23:19:32.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:32 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:32.319+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:32.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:32 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:32.319+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:32.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:32 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:32.319+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:32.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:32 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:32.319+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:32.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:32 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:32.289+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:32.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:32 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:32.289+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:32.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:32 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:32.289+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:32.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:32 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:32.420+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:32.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:32 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:32.420+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:32.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:32 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:32.420+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:32.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:32 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:32.420+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:32.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:32 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:32.420+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:33.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:33.156+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:33.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:33.156+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:33.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:33.156+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:33.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:33.026+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:33.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:33.026+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:33.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:33.026+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.882839+0000 front 2024-04-24T23:19:02.882739+0000 (oldest deadline 2024-04-24T23:19:27.582708+0000) 2024-04-24T23:19:33.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:33 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:33.345+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:33.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:33 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:33.345+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:33.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:33 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:33.345+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:33.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:33 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:33.345+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:33.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:33 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:33.271+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:33.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:33 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:33.271+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:33.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:33 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:33.271+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:33.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:33 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:33.271+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:33.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:33 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:33.391+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:33.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:33 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:33.391+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:33.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:33 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:33.391+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:33.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:33 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:33.391+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:33.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:33 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:33.391+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:34.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:33.997+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:34.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:33.997+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:34.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:33.997+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.882839+0000 front 2024-04-24T23:19:02.882739+0000 (oldest deadline 2024-04-24T23:19:27.582708+0000) 2024-04-24T23:19:34.319 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:33 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:33.997+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:10.384055+0000 front 2024-04-24T23:19:10.384095+0000 (oldest deadline 2024-04-24T23:19:33.184498+0000) 2024-04-24T23:19:34.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:34 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:34.127+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:34.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:34 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:34.127+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:34.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:34 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:34.127+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:34.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:34 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:34.127+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:34.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:34 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:34.383+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:34.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:34 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:34.383+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:34.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:34 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:34.383+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:34.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:34 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:34.383+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:34.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:34 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:34.313+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:34.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:34 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:34.313+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:34.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:34 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:34.313+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:34.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:34 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:34.313+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:34.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:34 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:34.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:34.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:34 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:34.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:34.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:34 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:34.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:34.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:34 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:34.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:34.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:34 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:34.414+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:35.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:35 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:35.002+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:43.278234+0000 front 2024-04-24T23:18:43.278255+0000 (oldest deadline 2024-04-24T23:19:07.378453+0000) 2024-04-24T23:19:35.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:35 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:35.002+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:52.480557+0000 front 2024-04-24T23:18:52.480711+0000 (oldest deadline 2024-04-24T23:19:17.180574+0000) 2024-04-24T23:19:35.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:35 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:35.002+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.882839+0000 front 2024-04-24T23:19:02.882739+0000 (oldest deadline 2024-04-24T23:19:27.582708+0000) 2024-04-24T23:19:35.318 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:35 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7[50332]: 2024-04-24T23:19:35.002+0000 7f1fa2048700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:10.384055+0000 front 2024-04-24T23:19:10.384095+0000 (oldest deadline 2024-04-24T23:19:33.184498+0000) 2024-04-24T23:19:35.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:35 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:35.103+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:35.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:35 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:35.103+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:35.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:35 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:35.103+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:35.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:35 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:35.103+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:35.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:35 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:35.388+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:35.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:35 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:35.388+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:35.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:35 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:35.388+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:35.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:35 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:35.388+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:35.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:35 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:35.362+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:35.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:35 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:35.362+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:35.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:35 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:35.362+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:35.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:35 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:35.362+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:35.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:35 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:35.373+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:35.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:35 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:35.373+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:35.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:35 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:35.373+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:35.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:35 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:35.373+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:35.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:35 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:35.373+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:36.406 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:36 smithi150 podman[66407]: 2024-04-24 23:19:36.118106005 +0000 UTC m=+5.487061370 container died fe1ff1c1a338d3791f7c94755d2d483992442f51fe7d9fe3ed3140bc311bdf4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T23:19:36.406 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:36 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:36.125+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:36.406 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:36 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:36.125+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:36.406 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:36 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:36.125+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:36.406 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:36 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:36.125+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:36.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:36.400+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:36.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:36.400+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:36.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:36.400+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:36.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:36.400+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:36.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:36.360+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:36.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:36.360+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:36.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:36.360+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:36.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:36 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:36.360+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:36.764 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:36 smithi150 podman[66407]: 2024-04-24 23:19:36.504264651 +0000 UTC m=+5.873220020 container cleanup fe1ff1c1a338d3791f7c94755d2d483992442f51fe7d9fe3ed3140bc311bdf4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-24T23:19:36.765 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:36 smithi150 bash[66407]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7 2024-04-24T23:19:36.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:36.353+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:36.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:36.353+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:36.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:36.353+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:36.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:36.353+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:36.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:36 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:36.353+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:37.068 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:36 smithi150 podman[66424]: 2024-04-24 23:19:36.763751361 +0000 UTC m=+0.648921222 container remove fe1ff1c1a338d3791f7c94755d2d483992442f51fe7d9fe3ed3140bc311bdf4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:37.443 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:37 smithi150 podman[66515]: 2024-04-24 23:19:37.1232439 +0000 UTC m=+0.114244028 container create 3272b296731a0bde0ce6ca9868aac37d7dfb9b7b324b6c8ba2cb83024d7d66a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T23:19:37.444 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:37 smithi150 podman[66515]: 2024-04-24 23:19:37.027337065 +0000 UTC m=+0.018337204 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:37.444 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:37 smithi150 podman[66515]: 2024-04-24 23:19:37.367365474 +0000 UTC m=+0.358365600 container init 3272b296731a0bde0ce6ca9868aac37d7dfb9b7b324b6c8ba2cb83024d7d66a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:37.444 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:37 smithi150 podman[66515]: 2024-04-24 23:19:37.370516085 +0000 UTC m=+0.361516212 container start 3272b296731a0bde0ce6ca9868aac37d7dfb9b7b324b6c8ba2cb83024d7d66a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T23:19:37.444 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:37 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:37.168+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:37.444 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:37 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:37.168+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:37.444 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:37 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:37.168+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:37.444 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:37 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:37.168+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:37.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:37.423+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:37.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:37.423+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:37.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:37.423+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:37.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:37.423+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:37.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:37.346+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:37.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:37.346+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:37.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:37.346+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:37.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:37 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:37.346+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:37.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:37 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:37.366+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:37.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:37 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:37.366+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:37.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:37 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:37.366+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:37.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:37 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:37.366+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:37.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:37 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:37.366+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:37.818 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:37 smithi150 podman[66515]: 2024-04-24 23:19:37.442335134 +0000 UTC m=+0.433335276 container attach 3272b296731a0bde0ce6ca9868aac37d7dfb9b7b324b6c8ba2cb83024d7d66a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-24T23:19:37.818 INFO:journalctl@ceph.osd.7.smithi150.stdout:Apr 24 23:19:37 smithi150 podman[66515]: 2024-04-24 23:19:37.663496734 +0000 UTC m=+0.654496861 container died 3272b296731a0bde0ce6ca9868aac37d7dfb9b7b324b6c8ba2cb83024d7d66a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T23:19:38.313 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:38 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:38.135+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:38.313 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:38 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:38.135+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:38.314 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:38 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:38.135+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:38.314 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:38 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:38.135+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:38.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.7.service' 2024-04-24T23:19:38.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:19:38.373 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-24T23:19:38.373 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-24T23:19:38.374 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.8 2024-04-24T23:19:38.693 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:38.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:38.693 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:38.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:38.693 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:38.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:38.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:38 smithi187 systemd[1]: Stopping Ceph osd.8 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:38.455+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:38.455+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:38.455+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:38.455+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:38.692+0000 7f5ed382d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:38.692+0000 7f5ed382d700 -1 osd.8 98 *** Got signal Terminated *** 2024-04-24T23:19:38.694 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:38 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:38.692+0000 7f5ed382d700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:38.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:38.355+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:38.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:38.355+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:38.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:38.355+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:38.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:38.355+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:38.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:38 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:38.355+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:39.512 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:39 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:39.129+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:39.512 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:39 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:39.129+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:39.512 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:39 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:39.129+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:39.512 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:39 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:39.129+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:39.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:39.434+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:39.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:39.434+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:39.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:39.434+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:39.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:39.434+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:39.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:39.350+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:39.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:39.350+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:39.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:39.350+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:39.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:39 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:39.350+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:39.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:39 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:39.343+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:39.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:39 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:39.343+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:39.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:39 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:39.343+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:39.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:39 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:39.343+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:39.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:39 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:39.343+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:40.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:40 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:40.094+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:40.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:40 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:40.094+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:40.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:40 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:40.094+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:40.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:40 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:40.094+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:40.638 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:40.418+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:40.638 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:40.418+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:40.638 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:40.418+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:40.639 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:40.418+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:40.639 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:40.318+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:40.639 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:40.318+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:40.639 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:40.318+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:40.639 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:40 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:40.318+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:40.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:40.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:40.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:40.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:40.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:40.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:40.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:40.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:40.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:40 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:40.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:41.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:41 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:41.073+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:41.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:41 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:41.073+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:41.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:41 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:41.073+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:41.568 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:41 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:41.073+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:41.284+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:41.284+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:41.284+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:41.284+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:41.284+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:41.387+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:41.387+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:41.387+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:41.653 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:41 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:41.387+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:41.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:41.341+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:41.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:41.341+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:41.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:41.341+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:41.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:41.341+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:41.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:41 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:41.341+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:42.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:42.063+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:42.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:42.063+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:42.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:42.063+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:42.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:42 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:42.063+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:42.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:42.388+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:42.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:42.388+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:42.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:42.388+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:42.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:42.388+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:42.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:42.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:42.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:42.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:42.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:42.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:42.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:42.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:42.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:42.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:42.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:42 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:42.311+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:42.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:42 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:42.311+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:42.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:42 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:42.311+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:42.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:42 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:42.311+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:42.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:42 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:42.311+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:42.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:42 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:42.311+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:43.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:43 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:43.031+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:43.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:43 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:43.031+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:43.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:43 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:43.031+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:43.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:43 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:43.031+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:43.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:43.413+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.028991+0000 front 2024-04-24T23:18:46.029000+0000 (oldest deadline 2024-04-24T23:19:10.128847+0000) 2024-04-24T23:19:43.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:43.413+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:50.129183+0000 front 2024-04-24T23:18:50.129260+0000 (oldest deadline 2024-04-24T23:19:15.429413+0000) 2024-04-24T23:19:43.736 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:43.413+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:00.130531+0000 front 2024-04-24T23:19:00.130361+0000 (oldest deadline 2024-04-24T23:19:26.030309+0000) 2024-04-24T23:19:43.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:43.413+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:06.030698+0000 front 2024-04-24T23:19:06.030715+0000 (oldest deadline 2024-04-24T23:19:31.930742+0000) 2024-04-24T23:19:43.737 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8[52115]: 2024-04-24T23:19:43.413+0000 7f5ecfe21700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:16.031595+0000 front 2024-04-24T23:19:16.032583+0000 (oldest deadline 2024-04-24T23:19:43.032128+0000) 2024-04-24T23:19:43.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:43.292+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:43.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:43.292+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:43.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:43.292+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:43.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:43.292+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:43.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:43 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:43.292+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:43.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:43 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:43.317+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:43.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:43 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:43.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:43.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:43 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:43.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:43.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:43 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:43.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:43.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:43 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:43.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:43.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:43 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:43.318+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:44.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:44.000+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:44.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:44.000+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:44.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:44.000+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:44.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:44.000+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:44.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:44 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:44.000+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:44.334 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:44 smithi187 podman[67829]: 2024-04-24 23:19:44.079758692 +0000 UTC m=+5.521811395 container died cfcbb55bc72465d7a1b8c68a71a8de60921837bbcbcd43eaabacd336fb90fbb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:44.612 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:44 smithi187 podman[67829]: 2024-04-24 23:19:44.417850346 +0000 UTC m=+5.859903060 container cleanup cfcbb55bc72465d7a1b8c68a71a8de60921837bbcbcd43eaabacd336fb90fbb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T23:19:44.612 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:44 smithi187 bash[67829]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8 2024-04-24T23:19:44.612 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:44 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:44.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:44.612 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:44 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:44.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:44.612 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:44 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:44.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:44.612 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:44 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:44.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:44.613 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:44 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:44.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:44.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:44 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:44.305+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:44.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:44 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:44.305+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:44.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:44 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:44.305+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:44.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:44 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:44.305+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:44.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:44 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:44.305+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:44.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:44 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:44.305+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:44.986 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:44 smithi187 podman[67858]: 2024-04-24 23:19:44.610836407 +0000 UTC m=+0.528669203 container remove cfcbb55bc72465d7a1b8c68a71a8de60921837bbcbcd43eaabacd336fb90fbb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T23:19:45.296 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:45 smithi187 podman[67952]: 2024-04-24 23:19:44.935306991 +0000 UTC m=+0.017644248 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:45.296 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:45 smithi187 podman[67952]: 2024-04-24 23:19:45.078678804 +0000 UTC m=+0.161016049 container create 224c0fc48b870738df1b0e032939307eb22c879e6861dd80e74a2888dd20c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-24T23:19:45.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:45.020+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:45.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:45.020+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:45.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:45.020+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:45.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:45.020+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:45.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:45 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:45.020+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:45.587 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:45 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:45.294+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:45.587 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:45 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:45.294+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:45.587 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:45 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:45.294+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:45.587 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:45 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:45.294+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:45.587 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:45 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:45.294+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:45.587 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:45 smithi187 podman[67952]: 2024-04-24 23:19:45.298563987 +0000 UTC m=+0.380901237 container init 224c0fc48b870738df1b0e032939307eb22c879e6861dd80e74a2888dd20c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T23:19:45.588 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:45 smithi187 podman[67952]: 2024-04-24 23:19:45.301829867 +0000 UTC m=+0.384167109 container start 224c0fc48b870738df1b0e032939307eb22c879e6861dd80e74a2888dd20c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-24T23:19:45.588 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:45 smithi187 podman[67952]: 2024-04-24 23:19:45.398478399 +0000 UTC m=+0.480815652 container attach 224c0fc48b870738df1b0e032939307eb22c879e6861dd80e74a2888dd20c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:45.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:45 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:45.332+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:45.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:45 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:45.332+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:45.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:45 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:45.332+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:45.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:45 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:45.332+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:45.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:45 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:45.332+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:45.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:45 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:45.332+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:45.868 INFO:journalctl@ceph.osd.8.smithi187.stdout:Apr 24 23:19:45 smithi187 podman[67952]: 2024-04-24 23:19:45.586375241 +0000 UTC m=+0.668712493 container died 224c0fc48b870738df1b0e032939307eb22c879e6861dd80e74a2888dd20c1fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T23:19:46.193 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.8.service' 2024-04-24T23:19:46.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:19:46.230 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-24T23:19:46.231 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-24T23:19:46.231 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.9 2024-04-24T23:19:46.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:46 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:46.052+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:46.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:46 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:46.052+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:46.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:46 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:46.052+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:46.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:46 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:46.052+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:46.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:46 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:46.052+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:46.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:46 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:46.339+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:46.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:46 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:46.339+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:46.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:46 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:46.339+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:46.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:46 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:46.339+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:46.487 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:46 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:46.339+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:46.527 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 systemd[1]: Stopping Ceph osd.9 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:46.527 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.316+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:46.527 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.316+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:46.527 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.316+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:46.528 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.316+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:46.528 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.316+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:46.528 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.316+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:46.528 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.525+0000 7f307139a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:46.528 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.525+0000 7f307139a700 -1 osd.9 98 *** Got signal Terminated *** 2024-04-24T23:19:46.528 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:46 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:46.525+0000 7f307139a700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:47.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:47 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:47.009+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:47.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:47 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:47.009+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:47.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:47 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:47.009+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:47.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:47 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:47.009+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:47.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:47 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:47.009+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:47.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:47 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:47.298+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:47.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:47 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:47.298+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:47.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:47 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:47.298+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:47.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:47 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:47.298+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:47.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:47 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:47.298+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:47.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:47 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:47.301+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:47.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:47 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:47.301+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:47.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:47 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:47.301+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:47.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:47 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:47.301+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:47.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:47 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:47.301+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:47.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:47 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:47.301+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:48.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:48 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:48.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:48.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:48 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:48.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:48.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:48 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:48.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:48.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:48 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:48.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:48.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:48 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:48.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:48.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:48 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:48.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:48.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:48 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:48.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:48.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:48 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:48.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:48.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:48 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:48.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:48.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:48 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:48.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:48.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:48 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:48.324+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:48.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:48 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:48.324+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:48.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:48 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:48.324+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:48.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:48 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:48.324+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:48.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:48 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:48.324+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:48.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:48 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:48.324+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:49.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:49 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:49.013+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:49.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:49 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:49.013+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:49.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:49 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:49.013+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:49.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:49 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:49.013+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:49.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:49 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:49.013+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:49.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:49.337+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:49.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:49.337+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:49.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:49.337+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:49.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:49.338+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:49.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:49.338+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:49.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:49 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:49.338+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:49.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:49 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:49.325+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:49.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:49 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:49.325+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:49.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:49 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:49.325+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:49.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:49 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:49.325+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:49.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:49 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:49.325+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:49.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:49 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:49.325+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:50.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.036+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:50.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.036+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:50.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.036+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:50.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.036+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:50.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.036+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:50.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.036+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:50.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:50 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:50.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:50.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:50 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:50.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:50.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:50 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:50.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:50.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:50 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:50.306+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:50.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:50 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:50.307+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:50.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:50 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:50.307+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:50.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:50 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:50.285+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:50.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:50 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:50.285+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:50.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:50 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:50.285+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:50.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:50 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:50.285+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:50.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:50 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:50.285+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:50.780 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:50 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:50.285+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:51.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.994+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:51.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.994+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:51.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.994+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:51.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.994+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:51.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.994+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:51.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:50 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:50.994+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:51.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:51 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:51.256+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.097705+0000 front 2024-04-24T23:18:37.097645+0000 (oldest deadline 2024-04-24T23:19:01.197789+0000) 2024-04-24T23:19:51.530 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:51 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:51.256+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:45.799201+0000 front 2024-04-24T23:18:45.799257+0000 (oldest deadline 2024-04-24T23:19:11.699347+0000) 2024-04-24T23:19:51.530 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:51 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:51.256+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:51.700023+0000 front 2024-04-24T23:18:51.699963+0000 (oldest deadline 2024-04-24T23:19:15.799902+0000) 2024-04-24T23:19:51.530 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:51 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:51.256+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:02.701722+0000 front 2024-04-24T23:19:02.701783+0000 (oldest deadline 2024-04-24T23:19:28.001856+0000) 2024-04-24T23:19:51.530 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:51 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:51.256+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.002376+0000 front 2024-04-24T23:19:08.002428+0000 (oldest deadline 2024-04-24T23:19:32.102298+0000) 2024-04-24T23:19:51.530 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:51 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9[64871]: 2024-04-24T23:19:51.256+0000 7f3069986700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.903927+0000 front 2024-04-24T23:19:17.903731+0000 (oldest deadline 2024-04-24T23:19:42.003584+0000) 2024-04-24T23:19:51.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:51 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:51.272+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:51.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:51 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:51.272+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:51.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:51 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:51.272+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:51.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:51 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:51.272+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:51.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:51 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:51.272+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:51.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:51 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:51.272+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:52.186 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:51 smithi112 podman[91611]: 2024-04-24 23:19:51.911657559 +0000 UTC m=+5.490653582 container died bbb3aec91b7c733497562f09d593fea823dc50af73563af2a7d07ace6deb6f65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-24T23:19:52.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:52 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:52.017+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:52.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:52 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:52.017+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:52.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:52 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:52.017+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:52.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:52 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:52.017+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:52.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:52 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:52.017+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:52.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:52 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:52.017+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:52.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:52 smithi112 podman[91611]: 2024-04-24 23:19:52.359042462 +0000 UTC m=+5.938038528 container cleanup bbb3aec91b7c733497562f09d593fea823dc50af73563af2a7d07ace6deb6f65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T23:19:52.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:52 smithi112 bash[91611]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9 2024-04-24T23:19:52.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:52 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:52.308+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:52.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:52 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:52.308+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:52.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:52 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:52.308+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:52.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:52 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:52.308+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:52.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:52 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:52.308+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:52.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:52 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:52.308+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:52.985 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:52 smithi112 podman[91628]: 2024-04-24 23:19:52.626994083 +0000 UTC m=+0.718081721 container remove bbb3aec91b7c733497562f09d593fea823dc50af73563af2a7d07ace6deb6f65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-24T23:19:53.239 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:52 smithi112 podman[91719]: 2024-04-24 23:19:52.887361981 +0000 UTC m=+0.017262152 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:53.239 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:52 smithi112 podman[91719]: 2024-04-24 23:19:52.986031296 +0000 UTC m=+0.115931452 container create f2f99630d0249976ec3667845addd7175029c4bc9b023d611dd814f11ebb4ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-24T23:19:53.239 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:53 smithi112 podman[91719]: 2024-04-24 23:19:53.238427176 +0000 UTC m=+0.368327332 container init f2f99630d0249976ec3667845addd7175029c4bc9b023d611dd814f11ebb4ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:53.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.027+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:53.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.027+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:53.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.027+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:53.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.027+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:53.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.027+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:53.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.027+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:53.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:53 smithi112 podman[91719]: 2024-04-24 23:19:53.241998201 +0000 UTC m=+0.371898360 container start f2f99630d0249976ec3667845addd7175029c4bc9b023d611dd814f11ebb4ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:53.529 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:53 smithi112 podman[91719]: 2024-04-24 23:19:53.321719102 +0000 UTC m=+0.451619261 container attach f2f99630d0249976ec3667845addd7175029c4bc9b023d611dd814f11ebb4ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:53.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:53 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:53.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:53.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:53 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:53.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:53.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:53 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:53.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:53.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:53 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:53.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:53.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:53 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:53.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:53.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:53 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:53.336+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:53.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 24 23:19:53 smithi112 podman[91719]: 2024-04-24 23:19:53.528266264 +0000 UTC m=+0.658166422 container died f2f99630d0249976ec3667845addd7175029c4bc9b023d611dd814f11ebb4ad7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:19:54.151 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.9.service' 2024-04-24T23:19:54.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:19:54.192 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-24T23:19:54.192 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-24T23:19:54.192 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.10 2024-04-24T23:19:54.246 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.989+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:54.246 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.989+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:54.246 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.989+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:54.246 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.989+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:54.246 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.989+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:54.246 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:53 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:53.989+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:54.549 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:54 smithi150 systemd[1]: Stopping Ceph osd.10 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:54.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:54 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:54.347+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:54.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:54 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:54.347+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:54.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:54 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:54.347+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:54.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:54 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:54.347+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:54.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:54 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:54.347+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:54.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:54 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:54.347+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:54.818 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:54 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:54.548+0000 7fc3675d9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:54.818 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:54 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:54.548+0000 7fc3675d9700 -1 osd.10 98 *** Got signal Terminated *** 2024-04-24T23:19:54.818 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:54 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:54.548+0000 7fc3675d9700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:55.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:55.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:55.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:55.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:55.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:55.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.005+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:55.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:55 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:55.379+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:55.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:55 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:55.379+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:55.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:55 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:55.379+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:55.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:55 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:55.379+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:55.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:55 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:55.379+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:55.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:55 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:55.379+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:56.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.983+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:56.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.983+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:56.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.983+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:56.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.983+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:56.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.983+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:56.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:55 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:55.983+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:56.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:56 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:56.386+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:56.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:56 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:56.386+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:56.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:56 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:56.386+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:56.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:56 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:56.386+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:56.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:56 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:56.386+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:56.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:56 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:56.386+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:57.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:57 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:57.014+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:57.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:57 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:57.014+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:57.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:57 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:57.014+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:57.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:57 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:57.014+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:57.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:57 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:57.014+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:57.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:57 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:57.014+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:57.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:57 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:57.014+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:30.971211+0000 front 2024-04-24T23:19:30.971188+0000 (oldest deadline 2024-04-24T23:19:56.871218+0000) 2024-04-24T23:19:57.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:57 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:57.351+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:57.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:57 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:57.351+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:57.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:57 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:57.351+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:57.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:57 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:57.351+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:57.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:57 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:57.351+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:57.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:57 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:57.351+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:58.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.003+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:58.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.003+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:58.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.003+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:58.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.003+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:58.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.003+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:58.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.003+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:58.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.003+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:30.971211+0000 front 2024-04-24T23:19:30.971188+0000 (oldest deadline 2024-04-24T23:19:56.871218+0000) 2024-04-24T23:19:58.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:58 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:58.373+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:58.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:58 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:58.373+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:58.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:58 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:58.373+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:58.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:58 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:58.373+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:58.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:58 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:58.373+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:58.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:58 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:58.373+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:19:59.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.978+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6804 osd.1 since back 2024-04-24T23:18:46.362596+0000 front 2024-04-24T23:18:46.362552+0000 (oldest deadline 2024-04-24T23:19:09.862818+0000) 2024-04-24T23:19:59.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.978+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.863650+0000 front 2024-04-24T23:18:49.863796+0000 (oldest deadline 2024-04-24T23:19:15.763305+0000) 2024-04-24T23:19:59.318 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.978+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:18:59.264335+0000 front 2024-04-24T23:18:59.264376+0000 (oldest deadline 2024-04-24T23:19:25.164453+0000) 2024-04-24T23:19:59.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.978+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:09.766115+0000 front 2024-04-24T23:19:09.766134+0000 (oldest deadline 2024-04-24T23:19:33.266230+0000) 2024-04-24T23:19:59.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.978+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:18.968428+0000 front 2024-04-24T23:19:18.968497+0000 (oldest deadline 2024-04-24T23:19:43.568962+0000) 2024-04-24T23:19:59.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.978+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:24.669997+0000 front 2024-04-24T23:19:24.669981+0000 (oldest deadline 2024-04-24T23:19:49.969922+0000) 2024-04-24T23:19:59.319 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:58 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10[57353]: 2024-04-24T23:19:58.978+0000 7fc35fbc5700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:30.971211+0000 front 2024-04-24T23:19:30.971188+0000 (oldest deadline 2024-04-24T23:19:56.871218+0000) 2024-04-24T23:19:59.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:59 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:59.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:19:59.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:59 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:59.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:19:59.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:59 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:59.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:19:59.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:59 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:59.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:19:59.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:59 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:59.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:19:59.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:19:59 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:19:59.332+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:20:00.159 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:19:59 smithi150 podman[66632]: 2024-04-24 23:19:59.874321073 +0000 UTC m=+5.482211560 container died b37882495cf4a863b883eee28731e41e8aca65cfca83754dce253c43d504530e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-24T23:20:00.550 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:00 smithi150 podman[66632]: 2024-04-24 23:20:00.265192909 +0000 UTC m=+5.873083386 container cleanup b37882495cf4a863b883eee28731e41e8aca65cfca83754dce253c43d504530e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:20:00.550 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:00 smithi150 bash[66632]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10 2024-04-24T23:20:00.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:00 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:00.319+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:20:00.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:00 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:00.319+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:20:00.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:00 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:00.319+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:20:00.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:00 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:00.319+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:20:00.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:00 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:00.319+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:20:00.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:00 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:00.319+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:20:00.818 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:00 smithi150 podman[66649]: 2024-04-24 23:20:00.549640513 +0000 UTC m=+0.666936217 container remove b37882495cf4a863b883eee28731e41e8aca65cfca83754dce253c43d504530e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-24T23:20:01.121 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:00 smithi150 podman[66737]: 2024-04-24 23:20:00.765259742 +0000 UTC m=+0.017852418 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:01.121 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:00 smithi150 podman[66737]: 2024-04-24 23:20:00.875506234 +0000 UTC m=+0.128116389 container create cbd16cae3aecf4c93e4d67890cb6b8c6bad05e3feb4c78033a44e0026d825a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T23:20:01.121 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:01 smithi150 podman[66737]: 2024-04-24 23:20:01.119850687 +0000 UTC m=+0.372443338 container init cbd16cae3aecf4c93e4d67890cb6b8c6bad05e3feb4c78033a44e0026d825a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:20:01.414 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:01 smithi150 podman[66737]: 2024-04-24 23:20:01.123163892 +0000 UTC m=+0.375756543 container start cbd16cae3aecf4c93e4d67890cb6b8c6bad05e3feb4c78033a44e0026d825a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:20:01.414 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:01 smithi150 podman[66737]: 2024-04-24 23:20:01.186103766 +0000 UTC m=+0.438696432 container attach cbd16cae3aecf4c93e4d67890cb6b8c6bad05e3feb4c78033a44e0026d825a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-24T23:20:01.699 INFO:journalctl@ceph.osd.10.smithi150.stdout:Apr 24 23:20:01 smithi150 podman[66737]: 2024-04-24 23:20:01.412543036 +0000 UTC m=+0.665135694 container died cbd16cae3aecf4c93e4d67890cb6b8c6bad05e3feb4c78033a44e0026d825a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:20:01.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:01.365+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:20:01.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:01.365+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:20:01.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:01.365+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:20:01.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:01.365+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:20:01.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:01.365+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:20:01.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:01.365+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:20:01.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:01 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:01.365+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:35.510072+0000 front 2024-04-24T23:19:35.510010+0000 (oldest deadline 2024-04-24T23:20:00.809860+0000) 2024-04-24T23:20:01.965 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.10.service' 2024-04-24T23:20:02.004 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:02.005 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-24T23:20:02.005 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-24T23:20:02.005 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.11 2024-04-24T23:20:02.322 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 systemd[1]: Stopping Ceph osd.11 for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:02.322 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.319+0000 7fc006839700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:02.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.319+0000 7fc006839700 -1 osd.11 98 *** Got signal Terminated *** 2024-04-24T23:20:02.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.319+0000 7fc006839700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:20:02.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.380+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:20:02.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.380+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:20:02.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.380+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:20:02.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.380+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:20:02.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.380+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:20:02.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.380+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:20:02.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:02 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:02.380+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:35.510072+0000 front 2024-04-24T23:19:35.510010+0000 (oldest deadline 2024-04-24T23:20:00.809860+0000) 2024-04-24T23:20:03.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:03.402+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:20:03.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:03.402+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:20:03.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:03.402+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:20:03.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:03.402+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:20:03.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:03.402+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:20:03.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:03.402+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:20:03.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:03 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:03.402+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:35.510072+0000 front 2024-04-24T23:19:35.510010+0000 (oldest deadline 2024-04-24T23:20:00.809860+0000) 2024-04-24T23:20:04.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:04.441+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:20:04.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:04.441+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:20:04.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:04.441+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:20:04.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:04.441+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:20:04.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:04.441+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:20:04.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:04.441+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:20:04.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:04 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:04.441+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:35.510072+0000 front 2024-04-24T23:19:35.510010+0000 (oldest deadline 2024-04-24T23:20:00.809860+0000) 2024-04-24T23:20:05.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:05.422+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:20:05.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:05.422+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:20:05.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:05.422+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:20:05.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:05.422+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:20:05.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:05.422+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:20:05.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:05.422+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:20:05.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:05.422+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:35.510072+0000 front 2024-04-24T23:19:35.510010+0000 (oldest deadline 2024-04-24T23:20:00.809860+0000) 2024-04-24T23:20:05.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:05 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:05.422+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6820 osd.8 since back 2024-04-24T23:19:40.810198+0000 front 2024-04-24T23:19:40.810227+0000 (oldest deadline 2024-04-24T23:20:05.410775+0000) 2024-04-24T23:20:06.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:06.406+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-24T23:18:37.999516+0000 front 2024-04-24T23:18:37.999552+0000 (oldest deadline 2024-04-24T23:18:59.699493+0000) 2024-04-24T23:20:06.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:06.406+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2024-04-24T23:18:49.001714+0000 front 2024-04-24T23:18:49.001686+0000 (oldest deadline 2024-04-24T23:19:16.002740+0000) 2024-04-24T23:20:06.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:06.406+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-24T23:19:01.204313+0000 front 2024-04-24T23:19:01.204264+0000 (oldest deadline 2024-04-24T23:19:24.704405+0000) 2024-04-24T23:20:06.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:06.406+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6812 osd.4 since back 2024-04-24T23:19:08.805596+0000 front 2024-04-24T23:19:08.805662+0000 (oldest deadline 2024-04-24T23:19:32.305514+0000) 2024-04-24T23:20:06.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:06.406+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2024-04-24T23:19:17.006666+0000 front 2024-04-24T23:19:17.006706+0000 (oldest deadline 2024-04-24T23:19:41.106856+0000) 2024-04-24T23:20:06.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:06.406+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-24T23:19:25.708288+0000 front 2024-04-24T23:19:25.708354+0000 (oldest deadline 2024-04-24T23:19:48.608089+0000) 2024-04-24T23:20:06.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:06.406+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.150:6820 osd.7 since back 2024-04-24T23:19:35.510072+0000 front 2024-04-24T23:19:35.510010+0000 (oldest deadline 2024-04-24T23:20:00.809860+0000) 2024-04-24T23:20:06.737 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:06 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11[58699]: 2024-04-24T23:20:06.406+0000 7fc002e2d700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.187:6820 osd.8 since back 2024-04-24T23:19:40.810198+0000 front 2024-04-24T23:19:40.810227+0000 (oldest deadline 2024-04-24T23:20:05.410775+0000) 2024-04-24T23:20:08.039 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:07 smithi187 podman[68058]: 2024-04-24 23:20:07.784929191 +0000 UTC m=+5.585115464 container died 0c6ba9e46c7413e4fe503e3425b289196b2335a288dea21b7b705f0dfa2b4d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T23:20:08.313 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:08 smithi187 podman[68058]: 2024-04-24 23:20:08.128259019 +0000 UTC m=+5.928445302 container cleanup 0c6ba9e46c7413e4fe503e3425b289196b2335a288dea21b7b705f0dfa2b4d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:20:08.314 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:08 smithi187 bash[68058]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11 2024-04-24T23:20:08.736 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:08 smithi187 podman[68076]: 2024-04-24 23:20:08.312664919 +0000 UTC m=+0.523717613 container remove 0c6ba9e46c7413e4fe503e3425b289196b2335a288dea21b7b705f0dfa2b4d4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T23:20:09.051 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:08 smithi187 podman[68171]: 2024-04-24 23:20:08.646821822 +0000 UTC m=+0.018233699 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:09.052 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:08 smithi187 podman[68171]: 2024-04-24 23:20:08.763652483 +0000 UTC m=+0.135064332 container create 0bfd32cba810773c226332528623a7a327207457f638bda8d58017cdf3b085c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T23:20:09.052 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:08 smithi187 podman[68171]: 2024-04-24 23:20:08.957574921 +0000 UTC m=+0.328986778 container init 0bfd32cba810773c226332528623a7a327207457f638bda8d58017cdf3b085c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True) 2024-04-24T23:20:09.052 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:08 smithi187 podman[68171]: 2024-04-24 23:20:08.960733604 +0000 UTC m=+0.332145465 container start 0bfd32cba810773c226332528623a7a327207457f638bda8d58017cdf3b085c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-24T23:20:09.052 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:09 smithi187 podman[68171]: 2024-04-24 23:20:09.050502267 +0000 UTC m=+0.421914135 container attach 0bfd32cba810773c226332528623a7a327207457f638bda8d58017cdf3b085c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:20:09.486 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:09 smithi187 podman[68171]: 2024-04-24 23:20:09.253431214 +0000 UTC m=+0.624843074 container died 0bfd32cba810773c226332528623a7a327207457f638bda8d58017cdf3b085c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T23:20:09.814 INFO:journalctl@ceph.osd.11.smithi187.stdout:Apr 24 23:20:09 smithi187 podman[68196]: 2024-04-24 23:20:09.794459941 +0000 UTC m=+0.533116387 container remove 0bfd32cba810773c226332528623a7a327207457f638bda8d58017cdf3b085c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-24T23:20:09.821 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@osd.11.service' 2024-04-24T23:20:09.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:09.862 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-24T23:20:09.863 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-24T23:20:09.863 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.a 2024-04-24T23:20:10.263 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:20:09 smithi112 systemd[1]: Stopping Ceph mds.a for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:10.529 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:20:10 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a[71721]: 2024-04-24T23:20:10.260+0000 7f389eb46700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:10.529 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:20:10 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a[71721]: 2024-04-24T23:20:10.260+0000 7f389eb46700 -1 mds.a *** got signal Terminated *** 2024-04-24T23:20:13.229 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:20:12 smithi112 podman[91837]: 2024-04-24 23:20:12.947451766 +0000 UTC m=+2.889800583 container died 6785c8969dea7c4c0901a5323ca88af5f1f846935d0613a193fc17cb5e68bb37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-24T23:20:13.529 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:20:13 smithi112 podman[91837]: 2024-04-24 23:20:13.319282724 +0000 UTC m=+3.261631529 container cleanup 6785c8969dea7c4c0901a5323ca88af5f1f846935d0613a193fc17cb5e68bb37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:20:13.529 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 24 23:20:13 smithi112 bash[91837]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-a 2024-04-24T23:20:13.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.a.service' 2024-04-24T23:20:13.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:13.795 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-24T23:20:13.795 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-24T23:20:13.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.d 2024-04-24T23:20:14.109 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 24 23:20:13 smithi112 systemd[1]: Stopping Ceph mds.d for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:14.529 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 24 23:20:14 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-d[70760]: 2024-04-24T23:20:14.106+0000 7f6e957bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:14.529 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 24 23:20:14 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-d[70760]: 2024-04-24T23:20:14.106+0000 7f6e957bc700 -1 mds.d *** got signal Terminated *** 2024-04-24T23:20:18.409 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 24 23:20:18 smithi112 podman[91959]: 2024-04-24 23:20:18.114580753 +0000 UTC m=+4.106736833 container died 70c3967d1086eb01f57c58a5e04f514b081e515bcf9619bab0441f41132586b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T23:20:18.742 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 24 23:20:18 smithi112 podman[91959]: 2024-04-24 23:20:18.489969017 +0000 UTC m=+4.482125103 container cleanup 70c3967d1086eb01f57c58a5e04f514b081e515bcf9619bab0441f41132586b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T23:20:18.742 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 24 23:20:18 smithi112 bash[91959]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-d 2024-04-24T23:20:18.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.d.service' 2024-04-24T23:20:18.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:18.903 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-24T23:20:18.903 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-24T23:20:18.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.g 2024-04-24T23:20:19.029 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:18 smithi112 systemd[1]: Stopping Ceph mds.g for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:19.529 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g[71363]: 2024-04-24T23:20:19.217+0000 7f8b6ada8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:19.529 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:19 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g[71363]: 2024-04-24T23:20:19.217+0000 7f8b6ada8700 -1 mds.g *** got signal Terminated *** 2024-04-24T23:20:21.262 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:20 smithi112 podman[92079]: 2024-04-24 23:20:20.949725325 +0000 UTC m=+1.835912420 container died 40f6a20ca81b0fd09dedee7bc05c2fb849b0ca508ff0ff81ab626fc3c9d1386c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-24T23:20:21.529 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:21 smithi112 podman[92079]: 2024-04-24 23:20:21.373056956 +0000 UTC m=+2.259244055 container cleanup 40f6a20ca81b0fd09dedee7bc05c2fb849b0ca508ff0ff81ab626fc3c9d1386c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:20:21.529 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:21 smithi112 bash[92079]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g 2024-04-24T23:20:21.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.g.service' 2024-04-24T23:20:21.832 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:21 smithi112 podman[92093]: 2024-04-24 23:20:21.578283638 +0000 UTC m=+0.622757951 container remove 40f6a20ca81b0fd09dedee7bc05c2fb849b0ca508ff0ff81ab626fc3c9d1386c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-24T23:20:21.833 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:21 smithi112 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.g.service: Deactivated successfully. 2024-04-24T23:20:21.833 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:21 smithi112 systemd[1]: Stopped Ceph mds.g for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:20:21.833 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 24 23:20:21 smithi112 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.g.service: Consumed 1.957s CPU time. 2024-04-24T23:20:21.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:21.844 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-24T23:20:21.844 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-24T23:20:21.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.j 2024-04-24T23:20:22.149 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:20:21 smithi112 systemd[1]: Stopping Ceph mds.j for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:22.529 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:20:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j[70412]: 2024-04-24T23:20:22.146+0000 7fc0151a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:22.529 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:20:22 smithi112 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j[70412]: 2024-04-24T23:20:22.146+0000 7fc0151a0700 -1 mds.j *** got signal Terminated *** 2024-04-24T23:20:24.002 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:20:23 smithi112 podman[92202]: 2024-04-24 23:20:23.736199152 +0000 UTC m=+1.684182371 container died b1feabf180444c65346d0759330c9c49d62a72a5906a2cbe900d4fc65c1d6795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:20:24.002 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:20:23 smithi112 podman[92202]: 2024-04-24 23:20:23.981108422 +0000 UTC m=+1.929091649 container cleanup b1feabf180444c65346d0759330c9c49d62a72a5906a2cbe900d4fc65c1d6795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:20:24.003 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 24 23:20:23 smithi112 bash[92202]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-j 2024-04-24T23:20:24.352 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.j.service' 2024-04-24T23:20:24.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:24.383 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-24T23:20:24.383 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-24T23:20:24.383 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.b 2024-04-24T23:20:24.684 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:24 smithi150 systemd[1]: Stopping Ceph mds.b for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:24.951 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:24 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: 2024-04-24T23:20:24.680+0000 7fa49249e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:24.952 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:24 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b[62631]: 2024-04-24T23:20:24.680+0000 7fa49249e700 -1 mds.b *** got signal Terminated *** 2024-04-24T23:20:29.027 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:28 smithi150 podman[66844]: 2024-04-24 23:20:28.695046316 +0000 UTC m=+4.116314479 container died 79bd8ab3f0b51dfbdee88e6516c04a67b1631e89e382c41971109419ea98c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-24T23:20:29.318 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:29 smithi150 podman[66844]: 2024-04-24 23:20:29.107088843 +0000 UTC m=+4.528357006 container cleanup 79bd8ab3f0b51dfbdee88e6516c04a67b1631e89e382c41971109419ea98c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T23:20:29.318 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:29 smithi150 bash[66844]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b 2024-04-24T23:20:29.585 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.b.service' 2024-04-24T23:20:29.606 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:29 smithi150 podman[66870]: 2024-04-24 23:20:29.34147423 +0000 UTC m=+0.646648944 container remove 79bd8ab3f0b51dfbdee88e6516c04a67b1631e89e382c41971109419ea98c011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-24T23:20:29.607 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:29 smithi150 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.b.service: Deactivated successfully. 2024-04-24T23:20:29.607 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:29 smithi150 systemd[1]: Stopped Ceph mds.b for e62ba378-028e-11ef-bc93-c7b262605968. 2024-04-24T23:20:29.607 INFO:journalctl@ceph.mds.b.smithi150.stdout:Apr 24 23:20:29 smithi150 systemd[1]: ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.b.service: Consumed 1min 15.234s CPU time. 2024-04-24T23:20:29.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:29.620 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-24T23:20:29.620 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-24T23:20:29.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.e 2024-04-24T23:20:29.938 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:20:29 smithi150 systemd[1]: Stopping Ceph mds.e for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:30.318 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:20:29 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e[62000]: 2024-04-24T23:20:29.936+0000 7facab0df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:30.318 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:20:29 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e[62000]: 2024-04-24T23:20:29.936+0000 7facab0df700 -1 mds.e *** got signal Terminated *** 2024-04-24T23:20:33.318 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:20:32 smithi150 podman[66977]: 2024-04-24 23:20:32.97067226 +0000 UTC m=+3.137651947 container died a40ffab21ec45c71c150b950fa469a5251db1106fcee7314d31bf247f0aa6f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:20:33.721 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:20:33 smithi150 podman[66977]: 2024-04-24 23:20:33.436003104 +0000 UTC m=+3.602982840 container cleanup a40ffab21ec45c71c150b950fa469a5251db1106fcee7314d31bf247f0aa6f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-24T23:20:33.721 INFO:journalctl@ceph.mds.e.smithi150.stdout:Apr 24 23:20:33 smithi150 bash[66977]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-e 2024-04-24T23:20:33.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.e.service' 2024-04-24T23:20:33.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:33.903 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-24T23:20:33.904 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-24T23:20:33.904 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.h 2024-04-24T23:20:34.318 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:20:33 smithi150 systemd[1]: Stopping Ceph mds.h for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:34.318 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:20:34 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h[62315]: 2024-04-24T23:20:34.222+0000 7f3b7ecbc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:34.318 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:20:34 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h[62315]: 2024-04-24T23:20:34.222+0000 7f3b7ecbc700 -1 mds.h *** got signal Terminated *** 2024-04-24T23:20:37.318 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:20:36 smithi150 podman[67098]: 2024-04-24 23:20:36.974801402 +0000 UTC m=+2.860534098 container died dfeb7077e80aed2d6324a6ba4cbc0b1302f77ad50f3e886d07738f7ce1defc6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:20:37.818 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:20:37 smithi150 podman[67098]: 2024-04-24 23:20:37.622642112 +0000 UTC m=+3.508374842 container cleanup dfeb7077e80aed2d6324a6ba4cbc0b1302f77ad50f3e886d07738f7ce1defc6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T23:20:37.818 INFO:journalctl@ceph.mds.h.smithi150.stdout:Apr 24 23:20:37 smithi150 bash[67098]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-h 2024-04-24T23:20:38.051 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.h.service' 2024-04-24T23:20:38.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:38.083 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-24T23:20:38.083 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-24T23:20:38.083 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.k 2024-04-24T23:20:38.318 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:20:38 smithi150 systemd[1]: Stopping Ceph mds.k for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:38.818 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:20:38 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k[61682]: 2024-04-24T23:20:38.401+0000 7f0e0ec5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:38.818 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:20:38 smithi150 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k[61682]: 2024-04-24T23:20:38.401+0000 7f0e0ec5a700 -1 mds.k *** got signal Terminated *** 2024-04-24T23:20:41.211 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:20:40 smithi150 podman[67220]: 2024-04-24 23:20:40.955109952 +0000 UTC m=+2.665017449 container died d0e0e2b7963d0a8ab3b89f07e9ff3bed2944263c450a567ba66058b2192c48b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:20:41.568 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:20:41 smithi150 podman[67220]: 2024-04-24 23:20:41.310073694 +0000 UTC m=+3.019981198 container cleanup d0e0e2b7963d0a8ab3b89f07e9ff3bed2944263c450a567ba66058b2192c48b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-24T23:20:41.568 INFO:journalctl@ceph.mds.k.smithi150.stdout:Apr 24 23:20:41 smithi150 bash[67220]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-k 2024-04-24T23:20:41.738 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.k.service' 2024-04-24T23:20:41.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:41.769 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-24T23:20:41.770 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-24T23:20:41.770 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.c 2024-04-24T23:20:42.071 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:20:41 smithi187 systemd[1]: Stopping Ceph mds.c for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:42.486 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:20:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c[63570]: 2024-04-24T23:20:42.069+0000 7fb710137700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:42.486 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:20:42 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c[63570]: 2024-04-24T23:20:42.069+0000 7fb710137700 -1 mds.c *** got signal Terminated *** 2024-04-24T23:20:45.236 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:20:44 smithi187 podman[68288]: 2024-04-24 23:20:44.967903077 +0000 UTC m=+3.002560726 container died 761366f0867225a49e85a1e3cac59f4068b017dd1fd3392a099b91b985c11c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:20:45.698 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:20:45 smithi187 podman[68288]: 2024-04-24 23:20:45.285134458 +0000 UTC m=+3.319792097 container cleanup 761366f0867225a49e85a1e3cac59f4068b017dd1fd3392a099b91b985c11c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T23:20:45.699 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:20:45 smithi187 bash[68288]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c 2024-04-24T23:20:45.699 INFO:journalctl@ceph.mds.c.smithi187.stdout:Apr 24 23:20:45 smithi187 podman[68300]: 2024-04-24 23:20:45.494527726 +0000 UTC m=+0.529249946 container remove 761366f0867225a49e85a1e3cac59f4068b017dd1fd3392a099b91b985c11c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T23:20:45.722 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.c.service' 2024-04-24T23:20:45.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:45.757 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-24T23:20:45.757 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-24T23:20:45.758 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.f 2024-04-24T23:20:45.986 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:20:45 smithi187 systemd[1]: Stopping Ceph mds.f for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:46.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:20:46 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f[63267]: 2024-04-24T23:20:46.072+0000 7fe30b108700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:46.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:20:46 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f[63267]: 2024-04-24T23:20:46.072+0000 7fe30b108700 -1 mds.f *** got signal Terminated *** 2024-04-24T23:20:49.223 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:20:48 smithi187 podman[68409]: 2024-04-24 23:20:48.950777041 +0000 UTC m=+2.975477557 container died ea1a12dc3cb2b06b8d2dd16b73fb056043723a993f227ab39e527f722b25f6bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-24T23:20:49.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:20:49 smithi187 podman[68409]: 2024-04-24 23:20:49.313470117 +0000 UTC m=+3.338170633 container cleanup ea1a12dc3cb2b06b8d2dd16b73fb056043723a993f227ab39e527f722b25f6bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:20:49.486 INFO:journalctl@ceph.mds.f.smithi187.stdout:Apr 24 23:20:49 smithi187 bash[68409]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-f 2024-04-24T23:20:49.737 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.f.service' 2024-04-24T23:20:49.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:49.768 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-24T23:20:49.769 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-24T23:20:49.769 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.i 2024-04-24T23:20:50.236 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:20:49 smithi187 systemd[1]: Stopping Ceph mds.i for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:50.236 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:20:50 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-i[63885]: 2024-04-24T23:20:50.076+0000 7f0c47595700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:50.236 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:20:50 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-i[63885]: 2024-04-24T23:20:50.076+0000 7f0c47595700 -1 mds.i *** got signal Terminated *** 2024-04-24T23:20:53.227 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:20:52 smithi187 podman[68531]: 2024-04-24 23:20:52.966526537 +0000 UTC m=+2.986437331 container died ea9686f71a196f3c6d03287da3eed76100744938c208aee3101fab041351f73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T23:20:53.486 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:20:53 smithi187 podman[68531]: 2024-04-24 23:20:53.342204298 +0000 UTC m=+3.362115101 container cleanup ea9686f71a196f3c6d03287da3eed76100744938c208aee3101fab041351f73e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-i, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:20:53.486 INFO:journalctl@ceph.mds.i.smithi187.stdout:Apr 24 23:20:53 smithi187 bash[68531]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-i 2024-04-24T23:20:53.723 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.i.service' 2024-04-24T23:20:53.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:53.754 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-24T23:20:53.754 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-24T23:20:53.754 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.l 2024-04-24T23:20:53.986 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:20:53 smithi187 systemd[1]: Stopping Ceph mds.l for e62ba378-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:54.486 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:20:54 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l[62963]: 2024-04-24T23:20:54.087+0000 7fea774bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:54.486 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:20:54 smithi187 ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l[62963]: 2024-04-24T23:20:54.087+0000 7fea774bd700 -1 mds.l *** got signal Terminated *** 2024-04-24T23:20:57.230 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:20:56 smithi187 podman[68651]: 2024-04-24 23:20:56.955969094 +0000 UTC m=+2.986229169 container died 4e6c70c50489794861c7798d0cfe358101aa3ba04d6e99d8056e76c29f4d7e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:20:57.486 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:20:57 smithi187 podman[68651]: 2024-04-24 23:20:57.312475802 +0000 UTC m=+3.342735886 container cleanup 4e6c70c50489794861c7798d0cfe358101aa3ba04d6e99d8056e76c29f4d7e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T23:20:57.486 INFO:journalctl@ceph.mds.l.smithi187.stdout:Apr 24 23:20:57 smithi187 bash[68651]: ceph-e62ba378-028e-11ef-bc93-c7b262605968-mds-l 2024-04-24T23:20:57.695 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e62ba378-028e-11ef-bc93-c7b262605968@mds.l.service' 2024-04-24T23:20:57.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:20:57.726 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-24T23:20:57.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e62ba378-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:20:57.918 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:21:00.439 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e62ba378-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:21:00.629 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:21:03.070 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e62ba378-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:21:03.257 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:21:05.588 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:21:05.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:21:05.653 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:21:05.679 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T23:21:05.681 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi112/crash 2024-04-24T23:21:05.682 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:21:05.708 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:21:05.708 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:21:05.710 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi150/crash 2024-04-24T23:21:05.711 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:21:05.833 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:21:05.834 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:21:05.836 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi187/crash 2024-04-24T23:21:05.836 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:21:05.873 INFO:teuthology.orchestra.run.smithi187.stderr:tar: /var/lib/ceph/e62ba378-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:21:05.873 INFO:teuthology.orchestra.run.smithi187.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:21:05.875 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T23:21:05.875 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-24T23:21:05.909 INFO:tasks.cephadm:Compressing logs... 2024-04-24T23:21:05.909 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:21:05.952 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:21:05.955 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:21:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:21:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:: No such file or directory 2024-04-24T23:21:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:21:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:21:05.976 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-24T23:21:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-24T23:21:05.977 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T23:21:05.978 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 86.1% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log: /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.6.log: 94.7% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-24T23:21:05.984 INFO:teuthology.orchestra.run.smithi112.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:21:05.987 INFO:teuthology.orchestra.run.smithi187.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:21:05.987 INFO:teuthology.orchestra.run.smithi187.stderr:: No such file or directory 2024-04-24T23:21:05.987 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:21:05.987 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:gzip/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr: 86.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.2% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-24T23:21:05.988 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.5% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-24T23:21:05.994 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:21:05.994 INFO:teuthology.orchestra.run.smithi150.stderr:: No such file or directory 2024-04-24T23:21:05.994 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:21:05.994 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:21:05.994 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-24T23:21:05.994 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-24T23:21:05.994 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T23:21:05.994 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.7.log: /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.b.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr: 86.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr: 90.2% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.1% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.3% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-24T23:21:05.995 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-24T23:21:05.998 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-24T23:21:05.998 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:21:05.998 INFO:teuthology.orchestra.run.smithi112.stderr: 83.2% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:21:05.998 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-24T23:21:05.999 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:21:06.003 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-24T23:21:06.006 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-24T23:21:06.009 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:21:06.010 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-24T23:21:06.010 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-24T23:21:06.011 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-24T23:21:06.014 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:21:06.023 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-24T23:21:06.037 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:21:06.191 INFO:teuthology.orchestra.run.smithi112.stderr: 90.1% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-24T23:21:06.274 INFO:teuthology.orchestra.run.smithi112.stderr: 95.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-24T23:21:06.281 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-24T23:21:06.298 INFO:teuthology.orchestra.run.smithi112.stderr: 95.1% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-24T23:21:06.396 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.k.log: 92.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-24T23:21:06.404 INFO:teuthology.orchestra.run.smithi150.stderr: 95.3% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-24T23:21:06.827 INFO:teuthology.orchestra.run.smithi112.stderr: 90.8% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-24T23:21:06.876 INFO:teuthology.orchestra.run.smithi187.stderr: 95.3% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-24T23:21:08.617 INFO:teuthology.orchestra.run.smithi187.stderr: 95.5% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-24T23:21:08.638 INFO:teuthology.orchestra.run.smithi187.stderr: 93.7% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-24T23:21:09.048 INFO:teuthology.orchestra.run.smithi150.stderr: 93.9% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-24T23:21:09.491 INFO:teuthology.orchestra.run.smithi150.stderr: 93.7% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-24T23:21:10.050 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-24T23:21:10.112 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-24T23:21:10.170 INFO:teuthology.orchestra.run.smithi112.stderr: 92.0% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T23:21:10.320 INFO:teuthology.orchestra.run.smithi112.stderr: 91.7% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T23:21:10.541 INFO:teuthology.orchestra.run.smithi150.stderr: 92.3% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T23:21:10.660 INFO:teuthology.orchestra.run.smithi112.stderr: 92.2% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-24T23:21:10.662 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m4.697s 2024-04-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m18.394s 2024-04-24T23:21:10.663 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.609s 2024-04-24T23:21:10.801 INFO:teuthology.orchestra.run.smithi187.stderr: 91.7% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-24T23:21:10.855 INFO:teuthology.orchestra.run.smithi150.stderr: 92.4% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-24T23:21:11.185 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-24T23:21:11.445 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T23:21:12.724 INFO:teuthology.orchestra.run.smithi150.stderr: 91.7% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T23:21:12.975 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T23:21:13.512 INFO:teuthology.orchestra.run.smithi187.stderr: 93.4% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-24T23:21:13.514 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-04-24T23:21:13.514 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m7.545s 2024-04-24T23:21:13.514 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m33.997s 2024-04-24T23:21:13.514 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m1.018s 2024-04-24T23:21:30.356 INFO:teuthology.orchestra.run.smithi150.stderr: 93.2% -- replaced with /var/log/ceph/e62ba378-028e-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-24T23:21:30.358 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-04-24T23:21:30.358 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m24.388s 2024-04-24T23:21:30.358 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m50.740s 2024-04-24T23:21:30.358 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m1.571s 2024-04-24T23:21:30.358 INFO:tasks.cephadm:Archiving logs... 2024-04-24T23:21:30.359 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi112/log 2024-04-24T23:21:30.360 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:21:33.439 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi150/log 2024-04-24T23:21:33.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:21:40.589 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi187/log 2024-04-24T23:21:40.590 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:21:44.983 INFO:tasks.cephadm:Removing cluster... 2024-04-24T23:21:44.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e62ba378-028e-11ef-bc93-c7b262605968 --force 2024-04-24T23:21:45.174 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:21:45.459 DEBUG:teuthology.orchestra.run.smithi150:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e62ba378-028e-11ef-bc93-c7b262605968 --force 2024-04-24T23:21:45.649 INFO:teuthology.orchestra.run.smithi150.stdout:Deleting cluster with fsid: e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:21:45.956 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e62ba378-028e-11ef-bc93-c7b262605968 --force 2024-04-24T23:21:46.147 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: e62ba378-028e-11ef-bc93-c7b262605968 2024-04-24T23:21:46.446 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T23:21:46.446 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:21:46.466 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:21:46.483 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:21:46.499 INFO:tasks.cephadm:Teardown complete 2024-04-24T23:21:46.499 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T23:21:46.518 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T23:21:46.518 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:21:46.521 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:21:46.526 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:21:46.639 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T23:21:46.649 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T23:21:46.650 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:21:46.652 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:21:46.654 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:21:46.666 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:21:46.668 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:21:46.669 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:21:46.671 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi150.stdout:^- hv01.front.sepia.ceph.com 2 6 377 24 +1128us[+1127us] +/- 579ms 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 +78us[ +77us] +/- 62ms 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -87us[ -87us] +/- 70ms 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi112.stdout:^- hv01.front.sepia.ceph.com 2 7 377 86 +1154us[+1146us] +/- 578ms 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv02.front.sepia.ceph.com 3 7 377 85 +109us[ +101us] +/- 62ms 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -101us[ -101us] +/- 70ms 2024-04-24T23:21:46.672 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:21:46.673 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:21:46.673 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-04-24T23:21:46.673 INFO:teuthology.orchestra.run.smithi187.stdout:^- hv01.front.sepia.ceph.com 2 6 377 20 +1128us[+1128us] +/- 579ms 2024-04-24T23:21:46.674 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 +69us[ +69us] +/- 62ms 2024-04-24T23:21:46.674 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -93us[ -93us] +/- 70ms 2024-04-24T23:21:46.674 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:21:46.674 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T23:21:46.686 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T23:21:46.687 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T23:21:46.698 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:21:46.790 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:21:46.855 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:21:46.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:21:46.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:46.947 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:21:46.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:46.974 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:21:47.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:47.001 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T23:21:47.012 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T23:21:47.023 INFO:teuthology.task.internal:Duration was 1434.198833 seconds 2024-04-24T23:21:47.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T23:21:47.036 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T23:21:47.036 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:21:47.039 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:21:47.041 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:21:47.077 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:21:47.078 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:21:47.078 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:21:47.606 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T23:21:47.606 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-24T23:21:47.607 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:21:47.626 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-04-24T23:21:47.627 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:21:47.669 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-04-24T23:21:47.669 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:21:47.695 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T23:21:47.695 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:21:47.698 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:21:47.711 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:21:47.761 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T23:21:47.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:21:47.763 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:21:47.777 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:21:48.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T23:21:48.160 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T23:21:48.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:21:48.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:21:48.208 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:21:48.231 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T23:21:48.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:21:48.249 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:21:48.252 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:21:48.268 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-24T23:21:48.272 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-04-24T23:21:48.293 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-04-24T23:21:48.316 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:21:48.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:48.334 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:21:48.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:48.353 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:21:48.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:21:48.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T23:21:48.382 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T23:21:48.383 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi112 2024-04-24T23:21:48.383 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:21:48.424 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi150 2024-04-24T23:21:48.424 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:21:48.459 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672181/remote/smithi187 2024-04-24T23:21:48.461 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:21:48.496 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T23:21:48.496 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:21:48.499 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:21:48.501 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:21:48.544 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T23:21:48.558 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T23:21:48.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T23:21:48.569 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T23:21:48.569 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:21:48.572 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:21:48.574 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:21:48.584 INFO:teuthology.orchestra.run.smithi112.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:21 /home/ubuntu/cephtest 2024-04-24T23:21:48.586 INFO:teuthology.orchestra.run.smithi150.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:21 /home/ubuntu/cephtest 2024-04-24T23:21:48.598 INFO:teuthology.orchestra.run.smithi187.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 23:21 /home/ubuntu/cephtest 2024-04-24T23:21:48.599 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T23:21:48.611 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T23:21:48.639 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/fs/norstats}} duration: 1434.1988334655762 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-24T23:21:48.639 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:21:48.723 INFO:teuthology.run:pass